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SUMMARY 


Objective 

Tin!  objective  wuh  information  concerning  the  organisation  of  pilots'  memories  for  critical  flight  related 
information  and  a  description  of  any  systematic  differences  in  memory  structure  that  were  related  to  differences  in 
flying  e*|H'rieiice. 


Background/Rationale 

Research  into  natural  language  has  suggested  that  retrieval  of  information  from  memory  is  affected  by  the 
organisation  of  memory.  More  rapid  and  effective  retrieval  can  result  from  a  more  efficient  and  economical  storage 
structure.  The  organisation  of  information  in  memory  can  have  a  critical  impact  on  flying  performance. 
Understanding  how  critical  information  is  organised  in  memory  can  be  extremely  useful  to  training  program 
designers  and  evaluators  as  well  as  Instructors  and  others  interested  in  increasing  the  effectiveness  of  the  pilot- 
aircraft  system.  Knowledge  of  how  individuals  develop  systems  for  organising  critical  information  can  be  used  to 
tailor  training  Nystema  to  provide  students  the  conceptual  framework  that  will  lead  to  optimal  learning.  It  may  also 
provide  a  useful  selection  tool  by  allowing  instructors  to  determine  which  Individuals  have  mastered  the 
prerequisite  concepts  for  success  in  a  particular  training  program. 


Approach 

Two  seis  of  stimulus  materials  displayed  on  the  console  of  a  Terak  8510/A  microcomputer  were  presented  to 
four  groups  of  officersi  Air  National  Guard  pilots  (GPa),  Fighter  Lead-in  Instructor  pilots  (IPs),  recent 
Undergraduate  Pilot  Training  (UPT)  graduates,  and  Instructor  Weapons  System  Officers  (WSOs).  Three  itaiistiral 
techniques  (hierarchical  cluster  analysis,  multi-dimensional  sealing  (MDS)  and  general  weighted  network  were 
used  to  analyse  the  data  and  describe  (lie  cognitive  structure  of  the  groups  studied. 

Specifics 

Subjects  were  nine  A-7  GPs  from  the  Colorado  Air  National  Guard,  seven  Fighter  Lead-in  IPs,  four  Fighter 
Lcud-in  Instructor  WSOs,  and  17  recent  UPT  Graduate*.  The  GPs  each  had  In  excess  of  2000  hours  flying  lime. 
The  IP*  had  1200—4000  hours,  afIR  the  UPT  graduates  had  about  200  hour*.  Although  the  GPa  and  IPs  had 
roughly  similar  amounts  of  flying  lime,  their  experience  differed  in  that  the  GPs  had  little  instructor  experience 
while  the  IPs  had  relatively  less  operational  experience.  The  WSOs  had  800—  3000  hours. 

Two  conceptual  sets  were  investigated.  One,  the  low-angle  strafe,  was  related  to  only  a  single  maneuver.  The 
miter  set  dealt  with  a  class  of  maneuvers!  the  split-plane  maneuvers.  The  stimulus  sets  were  genetated  by  the 
experimenters  working  with  senior  IPs  at  Holloman  AFB.  Populations  of  flight  related  terms  were  assembled 
through  interviews  with  IPs  at  Holloman  AFB,  These  were  condensed  to  one  set  of  30  stimulus  items  for  split-plane 
and  30  for  alrafn. 

Subjects  performed  a  self-paced  rating  task  in  which  all  pair-wise  combinations  of  the  items  in  each  set  were 
rated  for  similarity.  Similarity  ratings  were  treated  as  distances  in  conceptual  space.  In  addition,  the  UPs  gave  a 
rating  of  their  familiarity  with  the  lorms.  This  was  done  to  ensure  that  the  UPT  graduates  had  at  least  a  minimal 
degree  of  familiarity  with  the  terms. 

Analyses  showed  the  more  experienced  pilots  to  have  conceptual  structures  that  were  better  developed,  more 
sophisticated,  and  more  economical  thin  the  UPT  graduates.  The  WSOs  were  also  found  to  have  conceptual 
structures  which  differed  from  those  of  the  pilots, 

A  pattern  recognition  algorithm  waa  applied  to  the  MDS  solutions  and  to  the  raw  rating  data  to  see  if  the 
groups  could  he  distinguished  on  the  basis  of  their  conceptual  structures.  This  program  searched  for  a  pattern  or 


i 


prototype  which  characterised  the  conceptual  structure  of  members  of  a  group.  The  group*  were  found  to  have 
different  conceptual  structures,  with  the  more  experienced  individual!  showing  a  slightly  greater  tendency  to 
clualer  about  the  group  prototype,  while  the  recent  UPT  graduates  tended  to  be  more  diverse. 


ConelMelone/ReeoanendaUon* 

I .  Pilots  do  have  measurable  cognitive  structures  for  remembering  and  recalling  flight  related  information. 

Cognitive  structures  show  measurable  differences  as  a  function  of  flying  experience.  Experienced  pilots 
cxhihil  morn  efficient  and  economical  organisation  of  flight  related  information  than  do  leas  experienced  piiota. 
Tlte  WSOs  showed  a  memory  alruclure  that  differed  from  that  of  the  piiota. 

.'I,  The  approach  uaod  in  the  present  research  can  provide  a  useful  tool  for  looking  at  differences  in 
individuals'  understanding  of  flying  tusks,  Sueli  a  tool  may  Itave  application  both  for  assessing  infiivldual 
differences  In  the  development  of  conceptual  undaratandlng  during  learning  and  for  lonkhig  r  the  *  Tfectlveneas  of 
training  program*  in  conveying  critical  flight  related  ooncepta  to  studenta. 


prototype  which  characterized  the  conceptual  structure  of  members  of  a  group.  The  groups  were  found  to  have 
different  conceptual  structures,  with  the  more  experienced  individuals  showing  a  slightly  greater  tendency  to 
cluster  about  the  group  prototype,  while  l he  recent  UPT  graduates  tended  to  be  more  diverse. 

ConclUsions/Recommendations 

1.  Pilots  do  have  measurable  cognitive  structures  for  remembering  and  recalling  flight  related  information. 

2.  .Cognitive  structures  show  measurable  differences  as  a  function  of  flying  experience.  Experienced  pilots 
exhibit  more  efficient  and  economical  organization  of  flight  related  information  than  do  less  experienced  pilots. 
The  WSOs  showed  a  memory  structure  that  differed  from  that  of  the  pilots. 

S.  Tlie  approach  used  in  the  present  research  can  provide  a  useful  tool  for  looking  at  differences  in 
individuals'  understanding  of  flying  tasks.  Such  a  tool  may  have  application  both  for  assessing  individual 
differences  in  the  development  of  conceptual  understanding  during  learning  and  for  looking  at  the  effectiveness  of 
training  programs  in  conveying  critical  flight  related  concepts  to  students. 
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INTRODUCTION 


In  the  past  deoade,  experimental  psychologists  have  generated  a 
considerable  body  of  theory  and  data  concerning  the  organization  and 
retrieval  of  knowledge  in  human  memory.  This  research  area  (which  has 
come  to  be  known  as  semantic  memory)  has  concentrated  largely  on  the 
study  of  natural  categories  and  their  members  (e.g.,  birds,  minerals, 
geological  formations).  One  of  the  first  theoretical  proposals  in  the 
area  was  suggested  by  Collins  and  Cuillian  (1969)  following  the  lead 
developed  by  Quill ian  (1969)  in  the  form  of  an  intelligent,  question¬ 
answering  computer  system. 

Two  Important  structural  principles  were  embodied  in  the  theoret¬ 
ical  analysis  offered  by  Collins  and  Quillian:  hierarchical  organiza¬ 
tion  and  cognitive  economy.  The  hierarchical  principle  refers  to  the 
proposal  that  oonoepts  are  stored  in  memory  as  nodes  in  a  network  with 
eaoh  node  having  labeled  links  to  other  nodes  that  represent  super¬ 
ordinate  oonoepts.  For  example,  the  node  representing  the  ooncept 
"robin"  would  have  a  particular  kind  of  link  (ISA)  to  the  node 
representing  the  oonoept  "bird."  The  hierarchical  soheme  requires 
that  eaoh  oonoept  only  be  connected  to  its  immediate  superordinate  and 
not  to  more  general  oonoepts  (e.g,,  robin  is  direotly  connected  to 
bird  but  not  to  animal).  The  hierarohy  also  provides  a  basis  Tor 
inferences  about  facts  not  learned  direotly.  If  the  structure 
contains  the  faots  that  "an  A  is  a  B"  and  "a  B  is  a  C",  then  it  can  be 
inferred  that  "an  A  is  a  C." 

The  principle  of  cognitive  eoonomy  refers  to  the  way  in  which 
properties  of  oonoepts  are  represented  in  the  memory  system,  In 
particular,  properties  are  stored  at  the  highest  possible  level  of  the 
hierarohy.  This  means  that  properties  pertaining  to  all  members  of  a 
particular  category  need  to  be  stored  only  onoe  with  a  link  to  the 
node  representing  the  oategory.  For  example,  the  property  "han  wings" 
would  be  stored  with  the  oonoept  bird  rather  than  with  each  particular 
type  of  bird.  Thus  there  is  an  economy  or  storage.  Both  principles 
are  illustrated  in  the  network  structure  shown  in  Figure  1. 

Collins  and  Quillian  (1969,  1970)  presented  evidence  favoring  the 
hierarchical  theory  of  human  memory  structure.  They  showed  that 
people  were  faster  in  verifying  true  sentences  relating  concepts  near 
in  the  hierarohy  (e.g.,  A  robin  is  a  bird.)  oompared  to  sentences 
relating  more  distant  oonoepts  (e.g.,  A  robin  is  an  animal.).  Similar 
results  were  found  for  sentences  asserting  property  relations .  Those 
sentences  were  also  verified  more  slowly  when  the  noun  and  the 
property  were  further  apart  in  the  hierarohy. 


THING 


Subsequent  researoh  has  led  to  some  important  qualifications  of 
the  original  theory  proposed  by  Collins  and  Quillian.  Conrad  (1972) 
showed  that  some  of  the  original  support  for  the  idea  of  cognitive 
economy  was  due  to  a  confounding  of  associative  strength  between  nouns 
and  properties.  In  particular,  properties  which  were  near  to  the 
nouns  in  the  hierarchy  tended  to  occur  with  those  nouns  more  frequent¬ 
ly  in  the  language.  When  associative  strength  was  controlled,  there 
was  no  evidence  of  a  hierarchical  distance  effect.  Also,  the  assump¬ 
tion  of  hierarchical  organization  has  met  some  difficulties.  In 
particular,  Rips,  Shoben,  and  Smith  (1973)  have  shown  that  distance  in 
a  logical  hierarchy  does  not  always  predict  response  time.  For 
example,  people  verify  the  sentence,  "A  dog  is  an  animal”  faster  than 
the  sentence,  "A  dog  is  a  mammal.”  Smith,  Shoben,  and  Rips  (19T1*) 
discuss  some  of  the  problems  associated  with  the  hierarchy  model,  and 
they  propose  an  alternative  model  using  semantic  features  as  the  basio 
unit  of  analysis.  Collins  and  Loftus  (1975)  present  a  defense  of  the 
hierarchy  model.  They  also  suggest  extensions  to  the  model  which  help 
to  compensate  for  some  of  its  failures. 

There  have  been  several  other  proposals  concerning  memory  struc¬ 
tures  in  addition  to  the  hierarchical  structure  proposed  by  Collins 
and  Quillian.  We  have  already  mentioned  the  featural  analysis  pro¬ 
posed  by  Smith  et  al.  Others  have  proposed  that  a  multidimensional 
spatial  representation  captures  much  of  the  organization  of  some 
conceptual  domains.  Shepard  (1963)  and  Kruskal  (1977)  have  investi¬ 
gated  the  applicability  of  multidimensional  spatial  representations 
for  a  number  of  oonoeptual  domains  with  some  encouraging  results.  It 
can  be  difficult  to  discover  the  identity  of  the  underlying  dimensions 
which  limits  the  value  of  the  analysis  in  some  oases.  However,  multi¬ 
dimensional  soaling  (MDS)  does  provide  a  metrlo  (distanoe  in  multi¬ 
dimensional  spaoe)  whioh  oan  be  valuable.  The  present  project  has 
investigated  memory  structure  using  the  spatial  methods  developed  by 
Shepard  and  Kruskal. 

There  have  also  been  proposals  that  constitute  generalizations  of 
the  hierarchical  model  proposed  by  Collins  and  Quillian.  One  such 
model,  proposed  by  Collins  and  Loftus  (1975),  assumes  that  concepts 
are  organized  as  nodes  in  a  network  but  the  organization  of  the 
network  is  not  necessarily  hierarchical.  Retrieving  information  from 
such  a  network  requires  the  ”activation"  of  particular  nodes. 

A  powerful  context  mechanism  in  models  of  memory  structure  is 
termed  "spreading  activation”  (Collins  A  Quillian,  1969;  Collina  & 
Loftua,  19755  Meyer  A  Sohvaneveldt,  1971,  1976;  Schvaneveldt  &  Meyer, 
1973;  Sohvaneveldt,  Durao,  4  Mukherji,  1982).  According  to  spreading 
activation  theory,  the  activation  of  a  node  in  the  network  leads  to  a 
spread  of  aotivation  to  other,  nearby,  nodes.  Beoause  of  the  organ¬ 
ization  of  the  network,  the  nearby  nodes  are  semantically  related. 
Thus,  spreading  activation  makes  related  oonoepts  more  readily  access¬ 
ible  once  a  particular  concept  has  been  activated. 
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While  much  of  the  theory  ooneerning  structure  and  process  in 
semantic  memory  has  been  based  on  network  structures,  there  have  been 
no  empirical  methods  available  to  generate  networks  from  data.  One 
major  accomplishment  of  the  present  projeot  has  been  the  development 
of  such  a  method.  The  general  weighted  network  (GWN)  algorithm  uses 
the  same  data  as  other  methods.  Thus,  the  various  structural  repre¬ 
sentations  can  be  compared  using  the  same  data  sets. 

Development  of  Conceptual  Structures 

One  approaoh  to  the  validation  of  oonoeptual  structures  involves 
demonstrating  an  orderly  development  of  the  struoturea  in  student 
pilots  as  they  gain  more  experience.  The  structures  derived  from  the 
student  data  will  be  oomparad  with  those  derived  from  instructors  to 
identify  aapeots  of  the  oonoeptual  structure  which  undergo  marked 
changes  with  training  and  aspecta  that  remain  relatively  stable. 

During  the  past  year,  we  have  investigated  the  development  of 
spatial  representations  and  of  networks  of  oonoepts.  Of  interest  is 
the  extent  to  whioh  students  oan  be  distinguished  from  instructors  on 
the  basis  of  their  oonoeptual  structures.  We  have  also  attempted  to 
identify  specific  oonaepts  and  relations  between  oonoepts  which 
distinguish  instructors  from  students. 

In  summary,  the  preaent  investigation  employs  several  methods 
that  produce  structural  descriptions  for  oonoepts  from  split-plane 
maneuvers  (Table  1)  and  from  the  low-angle  strafe  (Table  2).  We  have 
derived  multidimensional  spatial  representations ,  cluster  analyses 
(Johnson,  1967),  and  network  representations  for  these  maneuvers. 
Multidimensional  spatial  representations  show  the  looation  of  eaoh 
conoept  in  a  multidimensional  payahologioal  space  where  the  Euclidean 
distance  between  oonoepts  represents  psyohologioal  proximity.  Cluster 
analyses  produce  a  seriea  of  groupings  of  concepts,  reflecting  under¬ 
lying  psychological  categories  of  oonaepts.  Cluster  analyses  Impose  a 
hierarchical  oonatraint  on  the  groupings  suoh  that  smaller  groups  form 
a  subset  of  larger  groups.  In  a  network  representation,  eaoh  concept 
ocours  as  a  node  in  a  network,  and  psychological  distance  is  repre¬ 
sented  by  the  distance  between  nodes.  Network  representations  can 
reveal  several  different  kinds  of  organizations  including  hierarchies, 
oyoles,  and  ohalns  of  oonoepts.  The  validity  of  these  representations 
is  assessed  by  various  oriterla,  including  consistency  within  and 
between  individuals,  the  development  of  the  representations  with 
training  and  experience,  and  performance  in  speeded  identl fioation 
tasks. 


GENERATION  OF  STIMULUS  MATERIALS 


A  review  of  training  and  technical  order  publications  related  to 
tactical  flight  operations  and  tactical  aircraft  provided  a  potential 
set  of  stimulus  items.  One  member  of  the  research  team  was  familiar 
with  tactical  flight  operations  as  an  Air  Force  navigator,  and  h*>  was 
responsible  for  the  generation  of  preliminary  stimulus  sets  related  to 
critical  flight  information  and  aircraft  systems. 

Early  in  this  phase  of  researoh,  a  rooent  AFHRL  technical  report 
came  to  our  attention.  The  report  (Meyer,  Laveson,  Pape,  A  Edwards, 
1978)  included  the  identification  and  detailed  task  analyses  of 
selected  basio  taotioal  flight  maneuvers.  Of  particular  interest  wore 
the  following: 

1.  The  breakdown  of  taotioal  operations  Into  two  practical  cate¬ 
gories,  air-to-air  and  air-to-ground. 

2.  Results  of  analyses  which  identified  "highly  representative" 
maneuvers  from  eaoh  of  the  above  categories. 

3.  The  use  of  a  scenario  as  a  means  of  oolleoting  data  from 
experienced  aircrew  members. 

Baaed  on  the  Meyer  et  al.  report,  two  representative  maneuvers 
were  seleoted  for  further  investigation.  The  two  maneuvers,  low  yo  yo 
and  low-angle  strafe,  repreaenting  air-to-air  and  air-to-ground 
oategories,  respectively,  together  with  a  general  list  of  taotioal 
airoraft  systems,  were  seleoted  for  further  stimulus  generation  and 
validation  efforts. 

Interviews  ware  conducted  with  four  instructor  pilots  from 
Holloman  AFB,  The  interviews  were  oonduoted  informally  and  involved 
examination  of  general  oonoepts  related  to  air-to-air  versus  air-to- 
ground  operations  depending  on  the  specialties  declared  by  the  inter¬ 
viewees.  Conceptual  units,  words  and  phrases,  were  then  compiled  into 
preliminary  stimulus  lists.  Detailed  review  or  the  Meyer  et  al.  task 
analyses  provided  additional  atimulus  items  and  a  basis  for  elimin¬ 
ating  items  not  dlreatly  rolatod  to  the  target  maneuvers  selected  for 
further  study. 

It  beoame  apparent  that  within  each  selected  maneuver,  certain 
Items  on  the  liat  could  be  considered  assumptions  neoessary  to  limit 
the  scenario  to  a  particular  maneuver.  Certain  others  were  essential 
or  oentral  to  the  maneuver,  and,  still  others,  to  varying  degrees, 
were  only  "related"  to  the  maneuver  in  a  more  abstract  sense.  Accord¬ 
ingly,  the  preliminary  lists  were  broken  down  into  three  subsets  based 
on  beat  available  information.  These  lists,  together  with  an  outline 
of  airoraft  systems,  were  then  ready  for  validation. 
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The  validation  procesa  oonaiated  of  a  aeries  of  Interviews  with 
individual  taotioal  fighter  pilots  on  a  aeries  of  visits  to  Holloman 
AFB.  The  purposes  of  the  project  ware  explained  to  each  pilot  who  was 
then  asked  to  review  a  list  of  conoepts.  The  resulting  additions, 
deletions,  and  shifts  (e.g.  from  "related"  to  "essential"  subsets) 
were  iteratively  incorporated  in  suooeeding  interviews.  The  10  inter¬ 
views  conducted  in  this  phase  resulted  in  the  following: 

1  .  The  low  yo  yo  scenario  was  expanded  to  include  all  primary 
split-plane  maneuvers.  Pilot  experts  made  it  clear  that,  even  with 
multiple  scenario-related  restrictions,  a  real  world  situation  oould 
preoipitate  any  combination  of  split-plane  maneuvers  and  no  single  set 
of  restrictions  would  evolve  into  a  low  yo  yo  maneuver  exclusively. 
Sinoe  many  of  the  essential  oonoepts  established  in  the  list  were 
oommon  to  all  split-plane  maneuvers,  only  minor  expansion  of  the 
stimulus  set  was  required. 

2.  The  low  angle  straTe  maneuver  was  kept  limited  in  soope  and 
oould  bo  effectively  restricted  to  the  specified  maneuver  given  few 
saenario-related  assumptions. 

3.  The  aircraft  systems  list  was  trimmed  considerably.  Pilots 
indicated  that,  while  all  represented  subsystems  could  be  considered 
oritioal  in  some  sense,  knowledge  about  several  airoraft  systems  only 
became  oritioal  when  speoific  malfunctions  occurred  in  flight. 

4.  The  "basio  oonoepts"  subset  of  each  maneuver  was  refined  to 
inolude  30  items  consistent  with  praotioal  experimental  manipulation 
in  the  next  phases  of  the  projeot. 

5.  The  "assumptions"  subset  of  eaoh  list  was  restricted  to  less 
than  10  items  to  provide  a  manageable  soenario  description  during  the 
next  phases  of  data  collection. 

The  resulting  lists  of  stimulus  items  are  shown  in  Tables  1,  2, 
and  3. 
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Table  1.  SCENARIO:  SPLIT-PLANE  MANEUVERS 


Assumptions 
OFFENSIVE 
SINGLE  BANDIT 
COMMIT 


AGGRESSIVE 

KILL 

ENGAGED 


TALLY  HO 

EQUAL  OR  SIMILAR  AIRCRAFT 
IR  MISSILE  PARAMETERS 


DEFENSIVE  TURN  (TARGET  DENIES  MISSILE) 


Baaio  Conospts 

LOW  YO  YO  HIGH  YO  YO  QUARTER  PLANE 

LAG  ROLL  BARREL  ROLL  OVERTAKE 

GUNS  AIRSPEED  ANGLE  OFF 

G  LOADING  CUTOFF  RELATIVE  ENERGY 

6  O'CLOCK  SMASH  POWER  SETTING 

SWITCHOLOGY  ACCELERATION  RADIAL  G 

HEAT  SNAPSHOT  VERTICAL  MANEUVERING 

3-9  LINE  EXTENSION  WEAPONS  PARAMETERS 

LAO  PURSUIT  LIFT  VECTOR  CORNER  VELOCITY 

ASPECT  ANGLE  PURE  PURSUIT  LEAD  PURSUIT 


Other  Related  Conoepts 

THREAT  DISENGAOE  BINGO 

JOKER  SADDLED-UP  LOW  PK 

HIGH  PK  BURIED  NOSE  REVERSAL 

SHOOTER  LOAD  PADLOCK 

RTB  EGG  OP  TURN 

MAX  TURN  TRAPPED  NOSE  LINE  OF  SIGHT  RATE 

NOSE  COUNTER  KNOCK-IT-OFF  OVER  PULL 

UNDER  PULL  SEPARATE  SUN 

CLOUDS 
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Table  2 


SCENARIO;  LOW  ANGLE  STRAFE 


Assumptions 

CONTROLLED  RANGE 
CLEARED 

Basic  Conoepts 
DRIFT 

GLIDE  PATH 

AIRSPEED 

BANK 

TRIGGER 

RICOCHET 

BURST 

STABILIZE 

RANGE 

BULLET  IMPACT 


PANEL/TARGET 

SWITCHOLOGY 


AIM  OFF  POINT 
FOUL  LINE 
RUN-IN  LINE 
PIPPER  FIXATION 
TRACKING 
YAW 

RECOVERY 

TRIM 

PIPPER  PLACEMENT 
GUNS 


TARGET  ACQUISITION 


DIVE  ANGLE 

CLOSURE 

ALTITUDE 

WALKING 

PULL  UP 

FINAL 

BUNT 

FOUL 

FIRE 

AIM  POINT 


ROLL  IN 

DOWNWIND  LEG 

PRESS 

SPACING 

BEARING 

TOWER 

COFFIN  CORNER 
EJECTION 


Other  Related  Conoepts 


RETICLE 

ANGLE  OF  ATTACK 
BASE  LEO 
SIGHT  PICTURE 
WINDS 
AZIMUTH 

DOWN-THE-CHUTE 
DOUBLE  BURST 
ROLLING  PULL 


CROSSWIND  LEG 

MIL  CRANKING 

WIRE 

TRACER 

REJOIN 

LAZY  PULL 

DISPERSION 

PENDULUM  EFFECT 

LONG  BURST 


Table  3.  ESSENTIAL  AIRCRAFT  TERMS 


FLIGHT-CONTROL  SYSTEM 
PITCH  DAMPER  SWITCH 
ADI 

CAS/MACH 

RUDDER  TRIM  KNOB 
THROTTLES 

FUEL-FLOW  INDICATORS 

COMM  RADIO 

RADAR  MISSILE 

GUN  COMPUTER 

HUD 

EJECTION  SYSTEM 


STICK 

FLIGHT-TRIM  SWITCH 

ALTIMETER 

RUDDER  PEDALS 

YAW  DAMPER  SWITCH 

FUEL-QUANTITY  INDICATORS 

NAVIGATION  SYSTEM 

FIRE-CONTROL  SYSTEM 

MISSILE  COMPUTER 

OPS  CHECKS 

CANOPY 

UHF 
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SIMILARITY-RELATEDNESS  RATINGS 


Method 

Moat  aoallng  procedures  for  producing  .structural  descriptions  of 
a  set  of  conaepta  require  some  measure  of  psychological  distance 
between  the  oonoepts.  Accordingly,  we  developed  a  similarity  or 
relatedness  rating  procedure  which  provided  measures  of  similarity 
between  the  members  of  all  possible  pairs  of  the  basic  concepts  from 
the  two  selected  scenarios  (see  Tables  1  and  ;>). 

A  program  was  developed  for  the  TERAK  microcomputer  to  colleot 
the  rating  data.  The  intent  was  to  produce  a  program  that  would 
essentially  allow  subjeots  to  perform  the  task  without  outside 
assistance.  Sinoe  testa  were  to  be  conducted  during  normal  duty 
hours ,  the  aim  was  to  establish  a  method  for  collecting  data  that 
would  impinge  on  subjeots'  time  as  little  as  possible.  Sinoe  the 
TERAK  could  be  continually  available  at  Holloman  AFB,  subjects  were 
able  to  run  through  our  programs  whenever  they  had  a  few  minutes  of 
free  time.  The  TERAK  was  programmed  to  automatically  start  the 
program  whenever  it  was  turned  on.  Posters  were  prepared  with 
detailed  instruotiona  about  starting  the  TERAK. 

Onoe  started,  the  "ERA;  presented  instructions  for  the  task  on 

its  video  display.  The  i^utruetions  described  the  nature  or  similar¬ 
ity  or  relatedness  ratings  and  gave  the  details  on  entering  the 
ratings  into  the  oomputer.  A  scenario  was  deaoribed  to  provide  a 
context  for  rating  the  terms,  and  the  complete  set  of  terms  to  be 

rated  waa  shown  to  allow  aubjeots  to  establish  some  oriteria  for 

rating  the  pairs  of  oonoepts.  The  rating  task  itself  oonsisted  of 
presenting  all  possible  pairs  of  the  30  basic  oonoepts  from  one  of  the 
sets  or  terms.  Thus  aubjeots  rated  the  similarity  of  435  pairs  of 
terms  during  the  seaaion.  For  eaoh  pair  of  terms,  the  TERAK  displayed 
the  pair  of  terms  to  be  rated,  a  rating  scale  with  the  numbers  0 
through  9,  and  a  bar  marker  to  indicate  the  rating.  Subjeots  entered 
their  rating  by  pressing  a  number  key  on  the  TERAK  keyboard.  The  bar 
marker  in  the  display  was  moved  to  the  position  corresponding  to  the 
number  entered  by  the  subject  to  indicate  the  rating  given.  Subjeots 
oould  ohenge  the  rating  by  pressing  another  number  key,  and  the  bar 
marker  would  move  to  the  position  corresponding  to  the  new  number. 
When  satisfied  with  the  rating,  the  subject  pressed  the  SPACE  BAR  on 
the  keyboard,  and  the  display  ohanged  to  show  the  next  pair  of  items 
and  to  reset  the  bar  marker  to  the  bottom  of  the  scale.  This  pro¬ 
cedure  was  followed  until  all  435  pairs  had  been  presented.  The  order 
of  the  pairs  was  Independently  randomized  for  each  subject,  A  rating 
session  required  from  30  to  45  minutes  to  complete,  and  the  TERAK  then 
presented  a  debriefing  to  the  subject  explaining  the  purpose  of  the 
researoh. 

Table  4  summarizes  the  subject  groups  that  were  tested  with  the 
rating  task.  Table  5  shows  the  flying  time  for  eaoh  subject. 
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Table  4.  Summary  of  Subject  Groupa  for  the  Rating  Task 


Group 

n 

Description 

Location 

Split-plane  Maneuvers 

IP 

7 

Instruotor  Pilots 

Holloman  APB 

GP 

9 

Air  National  Ouard  Pilots 

Buekley  ANOB 

UP 

17 

Undergraduate  Pilots  (UPTa) 

Williams  AFB 

IW 

4 

Instruotor  Weapons  Systems  Officers 

Holloman  &  Williams 

Low  Angle  Strafe  Maneuver 

IP 

6 

Instruotor  Pilots 

Holloman  AFB 

UP 

16 

Undergraduate  Pilots  (UPTs) 

Williams  AFB 

ivr 

7 

Instructor  Weapons  Systems  Offioera 

Holloman  &  Williams 

Table  5.  Hours  of  Flying  Time  for  Each  Subjeet 


Split-Plane  Maneuvers  Low  Angle  Strafe 


Subject 


Number 

IP(I) 

GP(0) 

UP(U) 

IW(W) 

IP(I) 

UP(U) 

IW(W) 

1 

2850 

3221 

182 

800 

2850 

200 

800 

2 

4300 

3500 

200 

950 

4300 

105 

970 

3 

3300 

2100 

105 

725 

1850 

200 

1500 

4 

2300 

21000 

200 

3000 

1400 

188 

1100 

5 

1230 

3850 

168 

4400 

175 

2100 

6 

1600 

10650 

175 

2600 

200 

725 

7 

4400 

4000 

200 

180 

3000 

8 

4000 

180 

206 

9 

2256 

206 

250 

10 

250 

200 

11 

200 

196 

12 

196 

300 

13 

300 

175 

14 

175 

173 

15 

173 

200 

16 

200 

275 

17 

275 

Note.  IP  -  Instructor  Pilots 

OP  -  Air  National  Guard  Pilots 
UP  -  Undergraduate  Pilots  (UPTs) 

IW  -  Instructor  Weapons  Systems  Offioers 
The  letters  shown  in  parentheses  next  to  the  group  labels 
are  used  to  designate  particular  individuals  In  the  group, 
e.g.,  II  represents  the  first  IP  listed. 


The  portability  of  the  research  apparatus  permitted  collection  of 
data  at  several  geographically  separated  locations  by  different 
researchers.  National  Guard  pilots  were  tested  at  the  Buckley  unit, 
of  the  Air  National  Guard.  Instructor  pilots  and  weapons  systems 
officers  were  tested  at  Holloman  AFB.  Undergraduate  Pilot  Training 
subjects  and  some  weapons  systems  officers  were  tested  at  Williams 
AFB. 

The  obtained  similarity  measures  were  transformed  into  measures 
of  psychological  distance  by  subtracting  the  ratings  from  the  maximum 
possible  rating.  The  resulting  numbers  reflect  distance  with  the 
larger  numbers  representing  greater  psyohologioal  distance  between 
oonoepta. 

The  rating  task  provided  the  data  base  for  much  of  the  work 
desoribed  in  this  report.  In  particular,  the  ratings  were  used  to 
evaluate  consistency  within  and  between  Individuals  and  within  and 
between  groups.  The  ratings  were  elso  used  to  produce  multidi¬ 
mensional  soaling  solutions,  hierarohioal  clusters,  networks,  and 
classification  algorithms,  The  details  for  eaoh  structural  analysis 
are  presented  in  the  appropriate  aeotions. 

Since  some  of  the  aubjeots  tested  were  in  UPT  training,  it  was 
desireable  to  determine  how  familiar  they  were  with  the  eonoepts  from 
the  two  lists.  Accordingly,  the  UPT  subjects  rated  their  familiarity 
with  the  terms  on  a  three  point  scale.  A  rating  of  1  indicated  that 
they  had  no  familiarity  at  all  with  the  concept.  A  reting  of  ? 
indicated  that  they  were  familiar  with  tha  term  but  did  not  use  it  in 
flying.  A  rating  of  3  indicated  that  tho  term  was  used  in  flying. 
Summaries  of  the  familiarity  rating  data  are  shown  in  Tables  6  and  7. 
Table  6  ahowa  the  familiarity  of  each  of  the  eonoepts  to  the  group  or 
UPTs  ea  a  whole.  Table  7  ahows  the  familiarity  indicated  by  eaoh  of 
the  UPTs  for  the  set  of  eonoepts  as  a  whole. 

As  oan  be  seen  from  the  tables,  the  UPTs  show  a  reasonable  degree 
of  familiarity  with  the  oonoepts.  Overall,  651  of  the  responses 
indicate  at  least  some  familiarity  with  the  split-plane  oonoepts,  and 
731  of  the  responses  indicate  at  least  some  familiarity  with  the  low- 
angle  strafe  oonoepta.  Obviously,  students  are  not  as  ramilisr  with 
the  oonoepts  as  instructors  (who  selected  the  oonoepta  to  begin 
with).  There  are  a  few  concepts  whioh  are  not  very  familiar,  but 
overall  the  students  appear  to  know,  or  at  least  know  about,  most  of 
the  oonoepts.  Also,  the  late"  analyses  suggest  that  there  are 
systematic  differences  between  UPTs  and  other  groups  whioh  are  more 
likely  based  on  systematic  misunderstanding  of  the  ooncepts  by  the 
UPTa  rather  than  a  laok  of  familiarity.  The  faot  that  some  of  the 
analyses  show  differences  between  Air  National  Guard  Pilots  and 
Instructor  Pilots  suggests  that  the  ability  to  discriminate  between 
groups  is  not  solely  based  on  a  lack  of  familiarity  in  one  of  the 
groups. 
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Table  6.  Familiarity  of  UPTs  with  Individual  Concepts 


Split-Plane  Maneuvers  Low-Angle  Strafe 


Familiarity  Rating  Familiarity  Rating 


Concept 

1 

2 

Concept 

1 

? 

3 

LOW  YO  YO 

10 

6 

_ 

0 

DRIFT 

0 

6 

9 

HIGH  YO  YO 

8 

8 

0 

AIM  OFF  POINT 

1? 

2 

1 

QUARTER  PLANE 

14 

2 

0 

DIVE  ANGLE 

1 

9 

5 

LAO  ROLL 

13 

3 

0 

GLIDE  PATH 

0 

3 

12 

BARREL  ROLL 

0 

1 

15 

FOUL  LINE 

14 

1 

0 

OVERTAKE 

0 

0 

16 

CLOSURE 

0 

5 

10 

OUNS 

1 

14 

1 

AIRSPEED 

0 

2 

13 

AIRSPEED 

0 

0 

16 

RUN-IN-LINE 

14 

1 

0 

ANGLE  OFF 

4 

8 

4 

ALTITUDE 

0 

1 

14 

0  LOADING 

0 

1 

15 

BANK 

0 

1 

14 

CUTOFF 

0 

0 

16 

PIPPER  FIXATION 

7 

8 

0 

RELATIVE  ENEROY 

0 

4 

12 

WALKING 

12 

3 

0 

6  O'CLOCK 

0 

3 

13 

TRIOOER 

2 

13 

0 

POWER  SETTING 

0 

0 

16 

TRACKING 

1 

10 

4 

SWITCHOLOGY 

7 

5 

4 

PULL-UP 

3 

5 

7 

ACCELERATION 

0 

1 

15 

RICOCHET 

6 

9 

0 

RADIAL  0 

12 

3 

1 

YAW 

0 

2 

13 

SMASH 

1 

1 

14 

FINAL 

2 

4 

9 

HEAT 

13 

3 

0 

BURST 

3 

12 

0 

SNAPSHOT 

13 

3 

0 

RECOVERY 

1 

8 

6 

VERTICAL  MANEUV 

0 

2 

14 

BUNT 

14 

1 

0 

3-9  LINE 

12 

4 

0 

STABILIZE 

2 

6 

7 

WEAPONS  PA RAMS 

4 

12 

0 

TRIM 

0 

1 

14 

CORNER  VELOCITY 

1 

8 

7 

FOUL 

14 

1 

0 

LIFT  VECTOR 

0 

4 

12 

RANGE 

1 

12 

2 

EXTENSION 

13 

3 

0 

PIPPER  PLACEMENT 

6 

9 

0 

ASPECT  ANGLE 

13 

3 

0 

FIRE 

1 

14 

0 

LAG  PURSUIT 

10 

4 

2 

BULLET  IMPACT 

3 

12 

0 

PURE  PURSUIT 

9 

5 

2 

GUNS 

1 

14 

0 

LEAD  PURSUIT 

10 

4 

2 

AIM  POINT 

1 

7 

7 

Total 

168 

115 

197 

121 

182 

147 

Percent 

35 

24 

41 

27 

40 

33 

Note.  Entries 

in  the 

table 

are 

the  number  of  UPTs  giving 

eaoh 

rating 

to  eaah  oonoept. 

1 - Totally  Unfamiliar 

2- Familiar  but  not  used  in  flying 

3- Used  in  flying 
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Table  7.  Familiarity  of  Individual  UPTa  with  All  Concepts 
Split-Plans  Maneuvers  Low-Angle  Strafe 


Familiarity  Ratings 


UPT 

1 

2 

3  . 

c 

— i 

U1 

- 1 

10 

"  14 

9 

13 

8 

U2 

12 

6 

12 

8 

13 

9 

U3 

12 

6 

12 

19 

3 

8 

U4 

15 

3 

12 

4 

10 

16 

U5 

12 

5 

13 

6 

14 

10 

U6 

4 

11 

15 

6 

16 

8 

U7 

7 

7 

16 

6 

13 

11 

U8 

10 

8 

12 

6 

10 

14 

U9 

13 

4 

13 

- 

- 

- 

U10 

_ 

- 

- 

7 

12 

11 

U11 

7 

13 

10 

9 

10 

1 1 

U12 

14 

10 

6 

8 

11 

11 

U13 

9 

9 

12 

8 

11 

1 1 

U14 

15 

3 

12 

11 

8 

1 1 

015 

11 

6 

13 

5 

25 

0 

U16 

5 

8 

17 

7 

12 

11 

017 

14 

6 

10 

Note.  Entries  in  the  table  ere  the  number  of 
oonoepts  given  each  rating  by  eaoh  UPT. 

1 - Totally  Unfamiliar 

2- Familiar  but  not  used  in  flying 

3- Uaed  in  flying 
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Results  and  Discussion 


The  first  step  toward  establishing  the  validity  of  the  rating 
data  requires  determining  ti.e  extent  to  which  different  subjects  agree 
about  the  ratings.  Under  the  assumption  that  the  cognitive  structures 
underlying  the  ratings  are  shared  by  people  with  similar  experiences, 
agreement  in  the  ratings  presumably  reflects  the  shared  structures. 
Table  8  shows  the  correlations  between  the  ratings  given  by  each  pair 
of  instructor  pilots.  Instructor  pilots  were  chosen  for  this  test 
since  they  presumably  have  an  organization  of  the  concepts  which  is 
communicated  to  the  students  they  are  training.  In  other  words, 
instructors  can  be  expected  to  have  a  reasonably  well  defined 
structure.  Correlating  the  ratings  they  give  to  the  various  pairs  of 
concepts  should  reflect  the  extent  to  which  they  share  a  common 
structure,  as  well  as  the  extent  to  which  the  ratings  succeed  in 
capturing  that  common  structure. 

The  correlations  between  members  of  a  pair  of  individuals  average 
about  .43  (the  average  of  the  off-diagonal  entries).  Given  the  large 
number  of  pairs  on  which  these  correlations  are  based,  they  are  all 
statistically  significant  (Critical  values  of  r  are  approximately  .10 
for  the  .05  level  of  significance  and  .13  for  the  .01  level  of 
significance).  The  obtained  correlations  suggest  a  moderate  amount  of 
agreement  among  the  instructor  pilots  in  the  ratings  assigned  to  the 
435  pairs  of  basic,  ooncepts  in  each  set  of  material. 

The  values  on  the  main  diagonals  (underlined)  in  the  matrioies 
in  Table  8  show  reliability  estimates  from  pilots  who  were  retested 
after  an  interval  of  6  to  8  months.  With  an  interval  that  long 
between  the  original  teat  and  the  retest,  the  second  set  of  ratings 
presumably  reflects  the  individual's  cognitive  structure  rather  than 
memory  for  the  ratings  given  on  the  initial  test.  The  reliability 
coefficients  average  about  .62  which  indicates  that  approximately  381 
of  the  varianoe  in  the  ratings  is  stable  over  time  within  an  indiv¬ 
idual.  The  average  correlation  between  individuals  cf  .43  leads  to 
the  conclusion  that  agreement  between  individuals  aooounts  for  about 
181  of  the  variance  in  the  ratings.  Putting  these  two  faots  together 
leads  to  the  conclusion  that  individuals  share  about  471  of  the 
consistent  varianoe  in  the  ratings  (18I/38I).  These  values  suggest 
that  the  agreement  between  individuals  is  not  only  statistically 
significant,  but  is  sufficiently  large  to  be  of  practical  significance 
as  well.  For  present  purposes,  the  rating  data  have  sufficient 
reliability  and  the  agreement  between  individuals  is  sufficiently  high 
to  allow  further  structural  analyses  of  these  data. 
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Table  8 


Inter-Individual  Correlation  Matrix  on  Bating  Scores 
II  through  17  are  individual  instructor  pilots 


Split-Piano  Manuevers 


11 

12 

13 

14 

15 

16 

17 

11 

m 

.35 

.47 

.45 

.42 

.58 

.43 

12 

.35 

- 

.41 

.41 

.47 

.38 

.32 

13 

.47 

.41 

- 

.49 

.43 

.42 

.43 

14 

.45 

.41 

.49 

- 

.41 

.43 

.41 

15 

.42 

.47 

.43 

.41 

- 

.42 

.41 

16 

.58 

•  38 

.42 

.43 

.42 

- 

.37 

17 

.43 

.32 

.43 

.41 

.41 

.37 

.67 

Low-Angle  Strafe  Maneuver 


11 

12 

13 

14 

15 

16 

11 

.49 

.39 

.57 

.49 

.41 

12 

.49 

- 

.40 

.50 

.42 

.39 

13 

.39 

.40 

.54 

.47 

.36 

.31 

14 

.57 

.50 

.47 

- 

.53 

.45 

15 

.49 

.42 

.36 

.53 

.71 

.42 

16 

.41 

.39 

.31 

.45 

753 

.54 
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HIERARCHICAL  CLUSTERING 


Figures  2  through  5  show  the  results  of  a  hierarchical  clustering 
analysis  of  the  basio  concepts  from  the  two  sets  of  oonoepta.  Figure 
2  shows  the  groupings  of  ooncepts  for  the  split  plane  maneuvers  for 
the  instructor  pilots,  and  Figure  3  shows  the  same  analysis  for  the 
undergraduate  pilots.  The  clusters  found  in  the  data  from  the  split- 
plane  oonoepts  for  both  instructor  pilots  and  undergraduate  pilots  are 
shown  in  Table  9.  In  several  of  the  methods  used  to  compare  students 
and  instructors,  there  are  areas  of  agreement  and  disagreement  in  the 
conceptual  structures.  For  example,  instructors  and  students  agree 
about  the  grouping  of  the  oonoepts  GUNS  and  SNAPSHOT.  However, 
students  group  HI  YO  YO  with  LO  YO  YO,  and  the  instructors  do  not  make 
suoh  a  grouping.  Perusal,  of  Table  9  will,  reveal  several  other 
examples. 

Similar  analysis  of  the  data  from  the  concepts  related  to  the  low- 
angle  strafe  are  shown  in  the  next  two  figures.  Figure  4  shows  the 
groupings  of  the  oonoepts  for  the  instructor  pilots,  and  Figure  5 
shows  the  results  for  the  undergraduate  pilots. 

In  general,  the  hierarohical  clustering  analysis  yields  sensible 
groupings  of  the  oonoepts,  especially  for  the  instructor  pilots. 
These  results  generally  confirm  the  validity  of  the  prooedures  we  have 
used.  The  duster  analysis  does  not  readily  yield  information 
permitting  more  detailed  analysis  than  the  concept  dusters 
themselves.  The  multidimensional  scaling  procedure  and  the  network 
analysis  have  proven  to  be  more  useful  in  pushing  the  analysis  of  the 
differences  between  students  and  instructors  to  a  more  detailed 
level.  Next,  we  turn  to  those  analyses. 
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RELATIVE  ENERGY 
POWER  SETTING 
ACCELERATION 


e  2.  Results  of  Hierarchical  Clustering  Analysis  of  Split  Plane  concepts  for  Instructor  Pilots. 


LIFT  VECTOR 
QUARTER  PLANE 
ANGLE  OFF 
3  9  LINE 
ASPECT  ANGLE 


Results  of  Hierarchical  Clustering  Analysis  of  Split  Plane  concepts  for  Undergraduate  Pilot  Trainees. 


STABILIZE 


i.  Results  of  Hierarchical  Clustering  Analysis  of  Strafe  concepts  for  Undergraduate  Pilot  Trainees. 


Table  9.  Major  Clusters  of  Split-Plane  Concepts 


Group  1 


Qroup  2 


Group  M _ Group  “5 


Instructor  Pilots 


LO  YO  YO  HI  YO  YO  G  LOADING  OUNS  ANGLE  OFF 

OVERTAKE  QTR  PLANE  VERT  MANEUV  SNAPSHOT  PURE  PURSUIT 

AIRSPEED  LAG  ROLL  RADIAL  G  CUTOFF  6  O'CLOCK 

SMASH  BARREL  ROLL  LIFT  VECTOR  LEAD  PUR  HEAT 

REL  ENERGY  LAO  PURSUIT  3-9  LINE  SWITCHOLOGY 

PWR  SETTING  ASPECT  ANQL  WEAPNS  PARMS 

ACCEL 

CORNER  VEL 
EXTENSION 


Undergraduate  Pilots 


LO  YO  YO  GUNS  SWITCHOLOGY  OVERTAKE  G  LOADINQ 


HI  YO  YO  SNAPSHOT 

LIFT  VECTOR  WEAPNS  PARMS 

QTR  PLANE  6  O'CLOCK 

ANGLE  OFF  HEAT 

3-9  LINE 

ASPECT  ANOL 

EXTENSION 

LAG  PURSUIT 

LAO  ROLL 

BARREL  ROLL 

RADIAL  0 


PURE  PUR  CORNER  VEL 

AIRSPEED 

REL  ENERGY 

SMASH 

PWR  SETTING 
ACCEL 
CUTOFF 
LEAD  PUR 
VERT  MANEUV 
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MULTIDIMENSIONAL  SCALING 


Struoturea.  The  results  of  two-dimensional  scaling  solutions  are 
shown  in  Figures  6  through  9.  Figure  6  shows  the  spatial  layout  of 
the  concepts  for  IPs  for  concepts  from  split-plane  maneuvers.  Figure 
7  shows  the  layout  generated  for  the  undergraduate  pilots  for  the  same 
set  of  concepts.  The  spatial  layouts  for  the  low-angle  strafe 
oonoepts  are  shown  in  Figures  M  and  9  for  the  instructor  pilots  and 
the  undergraduate  pilots,  respectively.  In  these  figures,  the 
position  of  the  concept  is  what  Is  being  represented.  The  horizontal 
position  represents  the  location  of  the  concept  on  one  dimension,  and 
the  vertioal  position  represents  the  looation  of  the  oonoept  on  a 
second  dimension. 

Slnoe  the  figures  only  represent  the  two-dimensional  sosling 
solutions,  they  do  not  capture  the  oomplete  struoturea  sinoe  more 
dimensions  are  required  to  represent  the  total  complexity  of  these 
sets  of  oonoepts.  Unfortunately,  solutions  with  more  than  two 
dimensions  are  difficult  to  represent.  However,  the  two-dimensional 
representations  in  the  figures  do  reveal  some  interesting  aspects  of 
the  conceptual  structures.  Comparing  the  instructors  with  the  under¬ 
graduates  (Figures  6  and  7),  for  example,  reveals  aome  similarities  in 
the  relative  looatlons  of  the  concepts  SWITCMOLOOY,  HEAT,  GUNS,  and 
WEAPONS  PARAMETERS  in  the  two  structures.  There  are  clear  differences 
in  the  relative  looationa  of  other  concepts.  Students  have  HI  YO  YO 
and  LO  YO  YO  looated  near  one  another  while  these  two  oonoepts  are 
muoh  further  apart  in  the  structure  developed  from  the  lnatruotor 
data.  We  will  return  to  a  detailed  analysis  or  the  differences 
between  experienced  and  inexperienced  pilots  in  some  of  the  later 
analyses. 

The  structures  we  used  for  further  analyses  were  based  on  five 
dimensions  for  the  split-plane  maneuvers  and  four  dimensions  for  the 
low-angle  strafe.  The  dimensionality  of  each  set  was  ohosen  by 
plotting  the  varianoe  in  the  instructor  data  whioh  is  accounted  for  by 
the  varianoe  in  the  Instructor  structures.  The  point  at  whioh  the 
funotion  began  to  level  off  was  selected  as  the  appropriate  dimension¬ 
ality  for  that  set  of  oonoepts.  Interestingly,  this  prooedure  leads 
to  a  more  oomplex  structure  for  the  split-plane  maneuvers  than  for  the 
low-angle  strafe.  Such  a  difference  would  be  expected  on  the  basis  of 
the  differenoe  in  complexity  for  the  two  sets  of  oonoepts. 

Multidimensional  aoallng  solutions  have  two  properties  that  are 
of  considerable  interest.  First,  they  provide  a  dimensional  organ¬ 
ization  that  can  reveal  interesting  global  structures  in  the  data. 
Second,  they  yield  a  metric  (distance  between  concepts  in  multi¬ 
dimensional  spaae)  whioh  has  some  useful  applications.  Next,  we  turn 
to  a  diaouasion  of  theae  properties  In  the  data  from  the  instructor 
pilots. 


Mu'  t  KS mensonai  Scaling  solution  of  Spl-l  Plane  concept r  for  Undergraduate  Pilot  Trail 


Dimensions  In  Multidimensional  Space.  Soma  of  the  major  dimen¬ 
sions  have  been  identified  for  the  spatial  layout  of  the  concepts  for 
instructors.  The  split-plane  concepts  have  two  dimensions  associated 
with  temporal  factors  and  one  dimension  which  distinguishes  particular 
maneuvers.  The  first  temporal  dimension  identifies  the  general  time 
dimension  within  a  scenario  leading  to  split-plane  maneuvers.  In 
Figure  6,  this  dimension  is  the  main  horizontal  dimension  ordered  from 
left  to  right.  The  concepts  on  the  extreme  left  (SWITCHOLOGY ,  HEAT, 
and  ANGLE  OFF)  refer  to  events  or  considerations  that  occur  early  in 
the  temporal  sequence.  Moving  to  the  right,  we  encounter  oonoepts 
referring  to  events  and  eona iderations  ooourring  later  in  the 
aequenoe.  The  second  temporal  dimension  represents  the  ordering  of 
oonoepta  In  a  standard  training  sequonoe  whioh  Instructors  and 
students  frequently  follow  in  praotiolng  the  maneuvers.  This 
dimension  oooura  as  the  third  dimension  in  ths  solution  and  is  not 
shown  in  Figura  6,  The  vertical  dimension  in  Figure  6  has  been 
identified  as  a  oontraat  betwaan  laad  pursuit  and  lag  pursuit  with  lag 
pursuit  and  tha  associated  maneuvers  near  the  top  and  lead  pursuit  and 
LOW  YO  YO  naar  the  bottom. 

The  low  angle  strafe  maneuver  also  provided  a  temporal  order 
dimension  as  the  first  dimension  In  the  solution.  Again,  this 
dimension  ooours  as  the  first  dimension  in  the  solution,  and  It 
refleota  the  order  in  whioh  the  oonoepta  would  ooour  to  piloto  in 
exeoutlng  the  low  angle  strafe.  Interestingly,  this  dimension  appears 
to  refleot  the  psychological  ordtring  of  the  oonoepts  rather  than  the 
order  in  whioh  the  events  occur  in  physical  time.  Apparently,  pilots 
must  oonaider  several  factors  asrly  in  time,  before  they  actually 
ooour,  in  order  to  be  able  to  aonoentrate  on  critioal  factors  such  as 
aiming  and  firing.  The  MDS  dimension  appears  to  refleot  this  order  of 
consideration.  This  dimension  appears  as  the  horizontal  dimension  in 
Figure  8  where  the  ordering  of  the  concepts  in  time  oooura  from  right 
to  left. 

These  dimension  are  summarized  in  Table  10.  Apparently  the 
temporal  order  dimension  is  a  powerful  one  in  the  organization  of 
these  oonoepts  for  pilots.  This  dimension  shows  up  consistently. 

le  dimensional  organization  of  the  oonoepts  la  interesting,  and  it 
lends  some  support  to  the  validity  of  the  analytic  procedures 
underlying  the  MDS  solutions.  However,  the  conclusions  drawn  so  far 
are  rather  general  and  of  limited  utility.  More  fine-grained  analyses 
of  the  structures  are  required  to  lead  to  oonolusions  that  may  by 
usefully  applied.  The  metrlo-based  analyses  will  take  us  a  step  In 
the  direction  of  applicable  findings. 
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Table  10.  Identity  of  the  Dimensions  In  Multidimensional  Spaoe 

Dimension  Identity  Conoept  Set 

1  Temporal  Order  of  Consideration  Split-plane 

2  Lead  versus  Lag  Pursuit  Split-plane 

3  Events  in  a  Training  Sequenoe  Split-plane 


1  Temporal  Order  of  Consideration  Low  Angle  Strafe 


The  MDS  Metric.  Table  11  shows  the  average  correlations  on  the 
split-plane  concepts  both  between  people  In  the  same  group  and  between 
Individuals  in  different  groups.  Table  12  shown  a  similar  analysis 
for  the  low-angle  strafe  concepts.  Several  aspects  of  the  data  in 
these  two  tables  are  noteworthy. 

First  note  that  the  least  experienced  individuals  (students) 
oonsistently  show  lower  correlations  both  within  their  own  group  and 
with  the  other  groups.  Overall  the  correlations  involving  students 
are  about  .23  oompared  to  an  average  for  the  other  groups  of  about 
.33.  Instructor  pilots  consistently  show  the  most  consistency  with 
members  of  their  own  group.  This  Is  perhaps  not  surprising  since 
instructors  not  only  know  about  the  execution  of  the  maneuvers,  but 
they  also  are  required  to  organize  what  they  know  so  they  can 
oommunioate  it  to  their  students.  Overall  the  correlations  suggest 
that  the  more  experienced  individuals  agree  more  among  themselves  than 
they  agree  with  the  students. 

On  the  other  hand,  the  students  do  show  a  reasonable  amount  of 
agreement  among  themselves.  Apparently,  whatever  it  is  that  they 
think  about  the  conoepts  being  rated,  they  share  the  same  knowledge  to 
some  extent. 

Finally,  note  that  the  Correlations  based  on  the  distances 
derived  from  the  multidimensional  scaling  are  slightly  but  oonsist¬ 
ently  higher  than  the  correlations  based  on  the  original  ratings. 
While  this  difference  is  not  large,  it  does  suggest  that  the  MDS 
dlstanoes  capture  at  least  as  much  structural  information  as  do  thr 
original  ratings.  As  we  will  see  later,  the  MDS  solutions  apparently 
do  contain  more  useful  information.  One  possible  explanation  for  this 
finding  is  that  the  MDS  procedure  involves  the  simultaneous  consid¬ 
eration  of  all  of  the  ratings  to  determine  the  best  relative  locations 
for  the  concepts.  The  original  ratings,  however,  only  require 
subjeots  to  consider  the  concepts  two-at-a-time .  Apparently,  the 
foroed  consistency  from  MDS  results  in  an  increase,  rather  than  a 
loss,  of  information.  The  pa  t  te  rn- reoogni  t  ion  analysis  (which 
follows)  dramatioally  demonstrates  the  superiority  of  the  MDS  metric 
over  the  original  ratings. 
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Table  11 


Average  Correlations  Within  Croups  and  Between  Groups 
Split-Plane  Maneuvers 


Scores 


Group 


IP 

GP 

UP 

IW 

Average 

IP 

.42 

.35 

.20 

.39 

.34 

OP 

.35 

.36 

.24 

.31 

.32 

UP 

.20 

.24 

.31 

.IB 

.23 

IW 

.39 

.31 

.18 

CO 

cn 

.32 

Dlatanoea  in  Multidimensional  Spaoe 
Croup 


IP 

CP 

UP 

IW 

Average 

IP 

.45 

.37 

.22 

.40 

.36 

ap 

.37 

.39 

.24 

.34 

.34 

up 

.22 

.24 

.29 

o 

ca 

• 

.24 

IW 

.40 

.34 

.20 

.38 

.33 

-32- 


Table  12 


Average  Correlations  Within  Groups  and  Between  Groups 
Low-Angle  Strafe  Maneuver 


Bating  Soores 

Oroup 


IP 

UP 

IW 

Average 

IP 

.44 

o 

CM 

• 

.39 

.34 

UP 

.20 

.32 

.24 

.25 

IW 

.39 

.24 

•  36 

.33 

PATTERN  RECOGNITION  ANALYSIS  OF  CONCEPTUAL  STRUCTURES 


Conceptual  structures  provide  relational  and  organizational 
information  about  the  oonoepts  of  a  particular  domain.  In  comparing 
structures  across  groups  of  Individuals  it  Is  possible  to  define 
qualitative  differences  between  groups.  This  type  of  a  comparison  is 
important  for  delineating  how  different  groups  view  a  particular  set 
of  concepts.  It  is  also  informative  to  express  differences  quantita¬ 
tively.  The  purpose  of  this  phase  of  the  project  is  to  define  a 
technique  for  quantitatively  evaluating  individual  and  group  differ¬ 
ences  in  conceptual  structures  of  critical  flight  information. 

The  objective  in  this  phase  of  the  project  is  the  development  of 
methods  for  classifying  an  individual  as  a  member  of  a  particular 
group  based  on  the  individual's  conceptual  structure.  For  instance, 
given  someone's  conceptual  structure  for  the  split-plane  maneuvers,  Is 
it  possible  to  identify  that  person  as  an  IP?  In  addition  to  classif¬ 
ication,  the  analysis  also  provides  information  about  the  degree  to 
which  each  individual  is  associated  with  each  group.  The  analysis  to 
be  described  applies  the  principles  and  techniques  of  pattern 
recognition. 

Pattern  recognition  is  an  area  of  artificial  intelligence  (AI) 
that  is  generally  oonoerned  with  deciding  whether  an  unknown  objeet  is 
a  member  of  a  particular  olass  of  objeots.  AT  applioations  often 
involve  computer  ldentifioation  of  visual  objeots,  although  numerous 
other  uses  exist.  The  only  prerequisite  for  the  application  of 
pattern  recognition  techniques  is  the  neoessity  to  quantify  the 
objeots  to  be  reoognized.  This  is  accomplished  by  first  identifying  a 
list  of  features  or  attributes  that  best  represent  the  objeots  and 
then  numerically  ooding  these  features.  Such  a  method  allows  for 
abstraot  as  well  as  physioal  objeots  to  be  analyzed.  In  addition  to 
categorizing  objeota  an  members  or  a  particular  olass,  pattern 
recognition  prinoiplea  alao  supply  information  about  olass  and 
individual  differences. 

Method 

Two  types  of  patterns  were  formed  for  each  individual  tested. 
One  pattern  waa  generated  from  the  conceptual  structures  derived  from 
a  multidimensional  scaling  (MDS)  analysis.  Eaoh  MDS  solution  yields  a 
metrio  formed  by  taking  the  distance  between  eaoh  pair  of  oonoepts  in 
a  multidimensional  spaoe.  The  MDS  pattern  was  oreated  by  viewing  the 
attributes  of  the  pattern  aa  values  of  the  metrio.  This  allowed  the 
pattern  to  preserve  the  structural  properties  inherent  in  the  MDS 
solution.  A  seoond  pattern  waa  generated  by  considering  the  similar¬ 
ity  rating  given  for  each  pair  of  oonoepts  as  a  feature  of  the 
pattern.  Since  this  pattern  was  simply  the  individual  similarity 
ratings,  it  laoked  the  structural  properties  imposed  by  scaling 
teohniquea.  Both  methods  resulted  in  patterns  with  4  35  features, 


Before  the  specific  analyses  are  described,  a  brief  overview  of 
the  theory  of  pattern  recognition  will  be  presented.  Nilsson  (1965) 
provides  a  general  discussion  of  the  principles  to  he  used  in  the 
projeot.  Objects  to  be  categorized  nre  represented  by  a  list  of 
feature  values  in  the  form  of  a  pattern  vector  X.  The  1th  element  of 
the  vector  X  represents  the  value  of  the  ith  feature.  Since  feature 
values  are  in  the  form  of  real  numbers,  pattern  vectors  can  be 
considered  as  points  in  a  multidimensional  space  where  each  dimension 
represents  an  attribute  of  the  object.  The  goal  is  to  develop 
deoision  surfaces  that  will  partition  the  pattern  space  into  regions 
containing  only  those  points  or  patterns  belonging  to  a  particular 
olass  of  patterns. 

One  way  of  produoing  deoision  surfaces  is  to  use  linear  discrim¬ 
inant  funotions  to  decide  olass  membership.  This  approaoh  assumes 
that  a  weighted  linear  combination  of  the  feature  values  can  determine 
how  a  pattern  should  be  olassified.  A  linear  discriminant  function 

has  the  form  g(X)»W1XHW2X2*. .  .-*.WdXd+Wd+1 ,  where  WsWI ,W2 . Wd  is  a 

vector  of  weights.  Suoh  a  function  speolfies  the  equation  of  a  line 
when  d«2  ,  the  equation  of  a  pla.,e  when  d«3  and  the  equation  of  a 
hyperplane  when  d>3.  Classes  that  oan  be  properly  separated  with 
linear  disoriminant  funotions  are  known  as  linearly  separable* 

Two  methods  for  generating  linear  disoriminant  funotions  will  be 
used.  The  first  method,  known  as  a  minimum-diatanoe  classifier,  is 
simple  to  apply  but  works  only  under  reatrioted  conditions.  With  this 
procedure,  a  prototype  point  representing  the  central  tendency  of  a 
olass  of  patterns  is  constructed  for  eaoh  pattern  olass.  Often  the 
prototype  point  of  a  olass  is  simply  the  average  of  the  feature  values 
of  all  patterns  belonging  to  the  class.  A  minimum-distanoe  classifier 
computes  the  dlstanoes  between  eaoh  pattern  to  be  categorized  and  eaoh 
prototype  point  and  places  the  pattern  into  that  class  associated  with 
the  nearest  prototype.  In  the  case  of  two  classes,  the  decision 
surfaoe  separating  the  patterns  is  the  perpendicular  biseotor  of  a 
line  connecting  the  two  prototype  points.  This  approach  works  well 
when  the  patterns  of  eaoh  class  cluster  tightly  around  their 
respective  prototype  points,  nnd  the  class  dusters  are  well 
separated, 

A  seoond  approach  to  pattern  classification  employs  a  training 
algorithm  that  alters  a  linear  disoriminant  function  until  it  correct¬ 
ly  classifies  the  patterns  in  a  training  set.  The  training  procedure 
alters  the  funotion  by  successive  adjustments  to  the  weight  vector  W 
which  in  effect  changes  the  orientation  and  position  of  the  decision 
surfaoe.  In  the  case  of  two  linearly  separable  classes,  a  weight, 
veotor  exists  that  will  produce  a  discriminant  funotion  that  returns  a 
positive  value  for  all  patterns  from  the  first  olass  and  a  negative 
value  for  all  patterns  from  the  second  class.  During  the  training 
procedure,  if  the  funotion  returns  a  correct  response  for  a  pattern 
from  the  training  set,  no  adjustment  is  made  to  the  weight  vector.  Tf 
the  funotion  returns  a  nogative  response  for  a  training  pattern  from 
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the  first  class,  the  weight  vector  is  corrected  by  adding  a  fraction 
of  the  pattern  vector  that  was  lncorr«otly  classified  to  the  weight 
vector.  This  produces  a  new  weight  vector  W'=W  +  cX,  where  c  is  a 
positive  number  that  controls  the  extent  of  the  adjustment  and  ia 
known  as  the  correction  increment.  II  c  is  large  enough,  the  new 
weight  veotor  will  correctly  classify  the  training  pattern.  If  the 
discriminant  function  incorrectly  returns  a  positive  value  for  a 
training  pattern  from  the  second  cl.'i.sa,  a  fraction  nr  the  pattern  ia 
subtracted  from  the  weight  vector  W'-W  -  nX. 

The  training  procedure  oonaista  of  presenting  the  training 
patterns  one  at  a  time  and  adjusting  the  weight  veotor  when  necess¬ 
ary.  The  patterns  may  be  presented  in  any  order  as  long  as  each 
pattern  is  tried.  The  procedure  is  terminated  an  soon  aa  the  weight 
veotor  oorreotly  olasslfies  all  patterns  in  the  training  set.  Several 
iterations  through  the  training  set  mny  he  necessary  before  a  solution 
is  found.  The  weight  vector  may  be  Initialised  to  any  convenient  set 
of  values  lnoluding  random  values. 

The  first  analysis  consisted  of  applying  a  minimum-distanoe 
olassifier  to  all  pairs  of  groups  for  both  the  split-plane  and  low- 
angle  strafe  maneuvers.  Prototype  points  for  all  groups  and  deoislon 
surfaoes  for  separating  all  pairs  of  groups  were  computed.  In  each 
application  of  the  miminum-di stance  classifier,  all  members  of  the  two 
groups  were  used.  This  provided  information  about  both  group  and 
individual  differences.  The  distances  from  oaah  individual  to  a 
deoislon  surface  and  from  eaoh  individual  to  the  group  prototypes  were 
oomputed  along  with  the  distances  between  group  prototypes. 

The  seoond  analysis  involved  computing  a  decision  surfsce  that 
separated  a  training  set  consisting  of  a  limited  number  of  members 
from  two  groups  and  then  applying  the  decision  surface  to  the  remain¬ 
ing  members  of  the  groups.  Decision  nurTaces  were  computed  with  a 
training  algorithm  if  a  minimum-distance  classifier  did  not  separate 
the  training  sets.  Weight  veotoru  were  initialized  to  the  weights 
produced  by  a  minimum-distance  classification  of  the  individuals  In 
the  limited  training  set,  This  minimized  the  number  of  Iterations 
naeded  to  produoe  a  solution  when  the  minimum  distance  olassifier 
failed  and  also  kept  the  final  weight  veotor  as  close  to  the  minimum- 
distanoe  deoislon  surface  as  possible.  The  first  analysis  showed  that 
the  classes  clustered  tightly  indicating  that  when  the  minimum- 
distanoe  weights  failed  to  separate  the  classes,  a  solution  olose  to 
these  weights  was  likely.  A  small  correction  Increment  (ca.01)  was 
used  to  produoe  minimal  change  from  the  minimum-distance  weights. 

For  eaoh  pair  of  groups,  a  training  set  of  a  particular  size  was 
randomly  ohosen,  and  a  decision  surface  wa3  computed  to  separate  the 
members  of  the  training  set  into  their  respective  classes.  In  the 
osse  where  a  minimum-distanoe  classifier  correctly  separated  the 
membera  of  the  training  sets,  the  resulting  discriminant  function  was 
then  applied  to  the  remaining  members  of  the  two  gt  oups.  If  no 


solution  was  found  with  the  minimum-distance  classifier,  the  training 
algorithm  was  applied  to  the  subset  of  selected  group  members  to 
generate  a  decision  surface  which  correctly  classified  all  individuals 
in  the  training  set.  The  resulting  discriminant  function  was  then 
applied  to  the  individuals  who  were  not  included  in  the  training  set. 
This  procedure  was  repeated  100  times  for  each  training  set  size.  The 
training  sets  consisted  of  equal  numbers  of  individuals  from  each 
group.  The  whole  procedure  was  iterated  with  successively  larger 
training  sets  until  the  size  of  the  smaller  group  was  reached. 

A  final  analysis  evaluated  group  differences  on  a  qualitative 
level.  Each  of  the  weights  in  a  discriminant  function  corresponds  to 
a  pair  of  concepts  in  the  stimulus  set.  Once  a  discriminant  function 
that  separates  two  groups  is  derived,  it  is  possible  to  identify  those 
concept  pairs  that  contribute  the  most  to  discriminating  between 
groups.  The  most  discriminating  pairs  of  concepts  correspond  to  the 
weights  in  the  weight  vector  with  the  greatest  absolute  values.  Large 
positive  weights  are  associated  with  pairs  of  concepts  that  the  first 
group  views  as  more  related  than  the  second  group,  and  large  negative 
weights  are  associated  with  pairs  of  concepts  that  the  second  group 
views  as  more  related  than  the  first  group.  Weight  vectors  derived 
from  a  minimum-distance  classification  of  the  MDS  patterns  were  used 
for  comparing  groups. 

Results  and  Discussion 

The  major  finding  was  that  pattern  recognition  techniques  can  be 
used  to  discriminate  classes  of  flying  personnel  based  on  their 
conceptual  structures  of  critical  flight  information.  Also  signif¬ 
icant  was  the  result  that  patterns  represented  by  distances  in  an  MDS 
solution  produced  better  group  separation  than  patterns,  based  on 
rating  scores. 

The  first  analysis  showed  that  a  minimum-distance  classifier 
applied  to  each  pair  of  groups  resulted  in  well  separated  groups  with 
only  a  few  erroneous  classifications.  Table  13  gives  the  number  of 
members  from  each  group  who  were  classified  into  each  of  the  groups. 
Incorrect  classifications  between  two  groups  indicate  that  the 
decision  surface  generated  for  those  groups  does  not  accurately 
separate  all  members.  Table  13  shows  the  classifications  for  both 
types  of  patterns  (ratings  and  MDS)  and  both  sets  of  stimulus 
materials  (split-plane  maneuvers  and  the  low-angle  strafe  maneuver). 

When  the  minimum-distance  classifier  was  applied  to  ratfhg  data, 
a  small  number  of  misclassifications  occurred.  For  the  split-plane 
maneuvers,  all  of  the  misclassifications  occurred  between  IPs  and 
UPTs.  The  two  IPs  who  were  classified  as  GPs  under  the  ratings  were 
on  the  GP  side  of  the  decision  surface  separating  IPs  and  GPs.  This 
means  that  their-  conceptual  structures  resembled  more  closely  those  of 
GPs  than  IP3.  Two  IPs  were  also  classified  as  IWSOs ,  and  we  will  see 
shortly  that  these  are  the  same  two  IPs  that  were  classified  as  GPs. 
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Table  13 


Classifications  Based  on  Croup  Separation 
with  a  Minimum-Distance  Classifier 


Split-Plane  Maneuvers 


Ratings 

Distances  in  MDS 

IP 

UP 

IWSO 

UTT 

IP 

QP  IWSO 

UPT 

IPs  classified  as 

3 

2 

2 

0 

7 

0  0 

0 

OPs  classified  as 

0 

9 

0 

0 

0 

9  0 

0 

IWSOs  classified  as 

0 

0 

4 

0 

0 

0  U 

0 

UPTs  classified  as 

0 

4 

1 

12 

0 

0  0 

17 

Low-Angle  Strafe 


Ratings 

Distances 

in  MDS 

IP 

IWSO 

UPT 

IP 

IWSO 

UPT 

IPs  classified  as 

6 

0 

0 

6 

0 

0 

IWSOs  classified  as 

0 

6 

1 

0 

7 

0 

UPTs  classified  as 

0 

3 

13 

0 

0 

16 
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These  two  individuals  view  the  split-plane  maneuvers  somewhat  differ¬ 
ently  from  the  other  IPs.  Table  13  also  shows  that  four  of  the  UPTs 
have  conceptual  structures  that  resemble  more  closely  a  GP  structure 
than  a  UPT  structure.  An  important  point  is  that  these  misclassi- 
fications  occur  even  though  the  individuals  misclassified  contributed 
to  defining  their  group  prototype. 

The  number  of  misclassifications  occurring  between  two  groups 
reflects  both  the  distance  between  the  groups  in  the  pattern  space  and 
also  the  tightness  with  which  individuals  cluster  around  their  respect¬ 
ive  prototypes.  IPs  and  UPTs  appear  to  be  very  distinct  classes  since 
no  IPs  were  classified  as  UPTs  and  no  UPTs  were  classified  as  IPs  for 
either  maneuver.  Considering  the  simplicity  of  the  decision  surfaces 
produced  by  the  minimum-distance  approach,  the  overall  results 
indicate  quite  distinct  classes  of  individuals. 

Table  13  also  provides  the.  results  of  the  minimum-distance  class¬ 
ifier  using  the  patterns  from  an  MDS  solution.  Here  we  find  perfect 
separation  of  all  classes.  Apparently  the  structural  information 
supplied  by  the  MDS  procedure  maximizes  the  differences  between 
classes.  This  finding  helps  to  validate  the  claim  that  the  MDS 
technique  extracts  important  structural  information  from  similarity 
ratings.  Additional  support  for  this  claim  comes  from  the  higher 
correlations  found  for  individuals  within  classes  using  MDS  distances 
in  comparison  to  rating  scores  (see  Tables  11  and  12). 

Although  the  number  of  misclassifications  reflects  between-group 
similarity,  a  more  direct  measure  is  the  distance  between  group 
prototypes.  Shorter  distances  suggest  greater  similarity  between 
group  conceptual  structures.  Table  1M  shows  these  distances  for  all 
pairs  of  groups  along  with  a  ranking  of  the  distances.  Given  the 
superior  performance  of  the  MDS  patterns,  the  ranking  based  on  the  MDS 
patterns  should  be  more  valid  than  that  of  the  ratings.  Table  1*J 
shows  that  the  most  similar  classes  are  IPs  and  GPs  followed  closely 
by  IPs  and  IWSOs.  Thus,  it  is  not  surprising  that  the  two  IPs  who 
were  misclassified  as  GPs  were  also  misclassified  as  IWSOs.  IPs  and 
UPTs  are  seen  to  be  two  of  the  most  dissimilar  groups  which  is  also 
consistent  with  the  finding  that  no  misclassifications  occur  between 
these  two  groups.. 

Sin  e  each  individual  is  represented  as  a  point  in  the  pattern 
space,  it  is  possible  to  provide  distances  that  reflect  how  similar 
the  individual  is  to  each  group.  Two  measures  are  of  particular 
interest,  the  distance  from  an  individual  to  a  decision  surface  and 
from  an  individual  to  the  group  prototype  points.  The  distance 
between  an  individual  and  the  decision  surface  separating  that 
individual’s  group  from  another  group  reflects  the  degree  to  which 
that  individual  belongs  to  the  group.  Large  distances  suggest  strong 
identification  with  the  group.  The  closer  the  individual  is  to  the 
decision  surface  the  more  similar  that  person  is  to  the  other  group. 
Negative  distances  indicate  that  the  person  i3'  on  the  wrong  side  of 
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Table  14 


Distances  Between  Group  Prototypes 
for  All  Pairs  of  Groups 


Split-Plane  Maneuvers 


Ratin£» 

Rank 

Distances  in  MDS 

Rank 

IP -OP 

33.36 

2 

81.30 

1 

IP-IWSO 

38.02 

4 

82.96 

2 

IP-UPT 

39.14 

5 

106. C9 

5 

OP-IWSO 

30.72 

1 

92.56 

4 

OP-UPT 

37.90 

3 

91.31 

3 

IWSO-UPT 

49.41 

6 

111.96 

6 

Low-Angle  Strafe 


i 

ten* 

Distances  in  MDS 

Rank 

IP-IWSO 

25.95 

1 

76.32 

1 

IP-UPT 

38.97 

3 

120.83 

3 

IWSO-UPT 

36.10 

2 

95.87 

2 
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the  decision  surface  and  is  therefore  misclassified.  The  distance 
between  an  individual  and  a  class  prototype  reveals  how  strongly  that 
individual  represents  the  average  features  of  that  class.  The  dist¬ 
ances  from  a  point  in  the  pattern  space  to  the  decision  surface  and 
from  the  point  t^  its  prototype  do  not  necessarily  correspond  because, 
for  example,  a  point  on  the  edge  of  the  pattern  space  may  be  close  to 

the  decision  surface  but  far  from  its  prototype. 

Tables  15  through  23  give  the  distances  from  an  individual  to 
both  decision  surfaces  and  group  prototypes  for  all  individuals.  The 

rating  patterns  for  IPs  in  Table  15  show  that  12  and  17  were  the  two 

IPs  who  were  incorrectly  classified  as  GPs  and  IWSOs.  It  is  interest¬ 
ing  to  note  that  17  is  in  fact  a  GP.  This  individual  was  placed  in 
the  IP  class  when  the  only  classes  available  were  IPs  and  IWSOs  and 
then  was  never  reclassified.  It  is  also  informative  to  examine  the 
number  of  hours  of  flying  time  for  both  12  and  17  from  Table  5  in 
comparison  to  the  other  IPs.  Both  have  considerably  more  hours  than 
do  the  other  IPs.  Their  fly'ng  time  is  actually  much  closer  to  the 
GPs'  flying  time. 

An  examination  of  the  MDS  patterns  from  Table,  15  shows  that 
although  all  IPs  were  correctly  classified,  12  and  17  have  the 
shortest  distances  to  the  decision  surface  separating  IPs  and  GPs. 
Generally,  there  is  good  correspondence  between  distances  for  the 
ratings  and  the  MDS  patterns.  When  disagreement  occurs,  it  must  be 
attributed  to  the  additional  information  supplied  by  MDS.  Again,  MDS 
should  provide  the  clearer  picture.  By  rank  ordering  the  distances 
under  the  GP  column,  it  is  possible  to  order  the  IPs  in  terms  of  their 
resemblance  to  GPs.  Similar  orderings  can  be  obtained  for  IPs  in 
comparison  to  IWSOs  and  UPTs.  The  distances  from  IPs  to  group 
prototypes  under  each  class  can  also  be  ordered.  As  seen  in  Table  15, 
13  most  closely  resembles  the  prototypical  IP  on  the  basis  of  concept¬ 
ual  structures  of  the  split-plane  maneuvers.  Similar  information  can 
be  obtained  for  the  members  of  the  other  classes. 

The  UPTs  are  particularly  interesting  since  they  are  currently 
undergoing  training  to  develop  expertise  In  flying  maneuvers.  From  the 
individual  distances  for  the  UPTs  in  Tables  18  and  19,  it  is  apparent 
that  some  of  the  UPTs  view  the  split-plane  maneuvers  more  like  the 
experts  than  do  the  other  UPTs.  For  instance,  07  was  classified  both 
as  a  GP  and  a  IWSO  in  addition  to  being  the  closest  UPT  to  the  IP 
decision  surface.  Although  no  raisclassifications  occurred  with  the 
MDS  patterns  the  same  trends  occur.  This  individual  appears  to  be 
approaching  the  conceptual  structure  of  the  experts  more  “quickly  than 
are  the  other  UPTs. 


Table  15 


Separation  of  IPs  from  Other  Groups 
Based  on  a  Minimum-Distanoe  Classifier 

Split-Plane  Maneuvers 
II  through  17  are  Individual  IPs 

Rating  Sooner 


Diatanoea  from  Distanoea  from 

Decision  Surface  Group  Prototypes 


OP 

IWSO 

UPT 

IP 

OP 

IWSO 

UPT 

11 

25.20 

34.25 

21.86 

<‘3.58 

59.84 

67.11 

60.09 

12 

— M  .13 

-6.50 

15.43 

44.14 

40.91 

38.13 

56.18 

13 

18.18 

18.45 

26.25 

41.30 

54.03 

55.76 

61.33 

14 

<49.29 

56.82 

14.40 

68.90 

89.64 

95.23 

76.65 

15 

9.12 

9.53 

15.99 

37.05 

44.51 

45.80 

51.24 

If 

32.30 

31.17 

34.25 

51.28 

69.19 

70.71 

72.88 

17 

-13.22 

-10.64 

8.83 

45.33 

34.25 

35.29 

52. 4P 

Average 

from  group 

prototypes: 

56.05 

58.29 

61.54 

Distanoea  in  Multidimensional  Space 


Distanoea  Yrom  Dlatanoei  from 

Deoision  Surfaoe  Oroup  Prototypes 


OP 

IWSO 

UPT 

IP 

GP 

IWSO 

UPT 

11 

34.52 

56.93 

52.72 

105.71 

129. 

57 

143.60 

149.60 

12 

32.22 

25.98 

39.83 

1 19.56 

139. 

76 

136.40 

150.87 

13 

44.09 

42.61 

72,90 

101.72 

132. 

35 

131.97 

160.76 

14 

52.87 

54.32 

59.18 

108.04 

142. 

37 

143.82 

155.73 

15 

39.83 

30.86 

42.47 

109. 18 

135. 

64 

130.54 

144.74 

16 

55.05 

43.91 

59.03 

114.96 

148. 

89 

143.18 

160.51 

17 

25.97 

35.76 

45.89 

111.97 

129. 

46 

135.91 

149.31 

Average 

from  group 

prototypes! 

110.16 

136. 

86 

137.92 

153.07 

Eaoh  row  of  the  table  represents  one  IP.  Deoision  surfaoes 
were  computed  for  separating  IPs  from  the  throe  remaining  groups. 
The  distanoe  from  each  IP  to  a  deoision  surface  is  shown  along 
with  the  diatanoe  from  eaoh  IP  to  eaoh  group  prototype.  Negative 
diatanoea  indioate  that  the  individual  is  on  the  wrong  side  of  the 
deoiaion  aurfaoe  and  therefore  misolassif led. 
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Table  16 


Separation  or  Gl’s  from  Other  Croups 
Bused  on  a  Minimum-Distance  Classifier 

Spl  it-Piano  Maneuvers 
01  through  G9  are  individual  Gia 

Rating  Snores 


Distances  from  Distances  from 

Deoision  Surfaoo  Oroup  Prototypes 


IP 

IWSO 

UPT 

IP 

OP 

IWSO 

UPT 

01 

17.51 

7.39 

24.20 

49.20 

35.39 

41.30 

55.56 

02 

7.68 

20.24 

10.16 

44.82 

38 . 69 

52.35 

47.61 

03 

6.53 

10.55 

13.06 

42.36 

36.85 

44.79 

48.45 

04 

11.04 

26.09 

5.41 

53.44 

46.04 

61.01 

50.29 

05 

6.58 

13.31 

2.10 

38.36 

32.13 

43.01 

34.52 

06 

35.50 

15.86 

35.12 

67.96 

47.38 

56.74 

70.05 

07 

13.67 

16.06 

24.43 

54.69 

45.45 

55.25 

62.59 

08 

30.94 

25.96 

29.69 

65.11 

46.64 

61.40 

66.52 

09 

20.40 

2.82 

26.37 

52.49 

37.33 

39.59 

58.24 

Average 

from  group 

prototypes! 

52.05 

40.66 

50.60 

54.87 

Diatanoos  in  Multidimensional  Space 


Distances  from  bistanoes  from 

Deaiaion  Surface  Oroup  Prototypes 


IP 

IWSO 

UPT 

IP 

OP 

IWSO 

UPT 

<31 

29.85 

34.75 

44.53 

135.06 

115. 

70 

140.78 

146. 

,69 

02 

44. B8 

46.49 

54.97 

145.52 

1 17. 

81 

149.96 

164, 

,65 

03 

24.01 

30.21 

57.31 

128.40 

112. 

71 

134 .82 

16I  , 

,82 

04 

50.97 

55.73 

29.34 

148.58 

117. 

4.? 

165.26 

138. 

36 

05 

38.91 

34.92 

33.20 

151.24 

128. 

64 

161 .70 

160. 

1 37 

06 

58.17 

65.18 

48.27 

157.95 

1?4, 

45 

166.00 

155. 

,H9 

07 

28.65 

43.15 

44.93 

132.55 

113. 

63 

144.56 

145. 

,31 

08 

70.97 

88.28 

43.82 

177.94 

141. 

85 

190.96 

167. 

,70 

09 

19.45 

17.83 

54.50 

131.07 

118. 

39 

131 .59 

154, 

,02 

Average 

from  group 

prototypes! 

145.37 

121. 

J8 

151.74 

151, 

.73 

Bach  row  of  the  table  represents  one  OP,  Decision  aurfaoos 
were  computed  for  separating  GPs  from  the  throe  remaining  groups. 
The  distance  from  eaoh  OP  to  a  deoision  surface  is  shown  along 
with  the  distanoe  from  each  OP  to  eaoh  group  prototype.  Negative 
distances  indioate  that  the  individual  is  on  the  wrong  side  of  the 
deoision  surface  and  therefore  misolnsstf’ied. 
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Table  17 


Separation  of  IWSOs  from  Other 
Groups  Based  on  a  Minimum-Distance  Classifier 

Split-Plane  Maneuvers 
W1  through  W4  are  individual  IWSOs 

Rating  Scores 


Distances  fVom  Distances  from 

Deoision  Surfaoe  Group  Prototypes 


IP 

OP 

DPT 

IP 

GP 

IWSO 

UPT 

W1 

36.26 

17.89 

37.30 

65.72 

51.59 

39.52 

72.45 

W2 

30.06 

17.24 

31.51 

60.47 

49.31 

37.03 

66.98 

W3 

5.65 

8.84 

10.80 

42.09 

43.43 

36.64 

49.09 

W4 

4.08 

17.74 

19.21 

49.73 

56.89 

46.50 

63.73 

Average 

from  group  prototypes: 

54.50 

50.31 

39.92 

63.06 

Distances  in  Multidimensional  Space 


Distanoes  from  Distances  from 

Deoision  Surface  Group  Prototypes 


IP 

GP 

UPT 

IP 

GP 

IWSO 

UPT 

W1 

53.90 

47.18 

63.20 

143.22 

142.50 

107.56 

160.38 

W2 

74.84 

73.47 

60.53 

167.57 

171.07 

125.15 

170.93 

W3 

21.36 

31.93 

43.14 

119.50 

129.02 

103.61 

142.81 

W4 

15.83 

32.55 

57.05 

119.52 

132.98 

107.97 

156.30 

Average 

from  group 

prototypes: 

137.45 

143.89 

111.07 

157.61 

Each  row  of  the  table  represents  one  IWSO.  Deoision  surfaoes 
were  oomputed  for  separating  IWSOs  from  the  three  remaining  groups. 
The  distanoe  from  eaoh  IWSO  to  a  decision  surface  is  shown  along 
with  the  distanoe  from  eaoh  IWSO  to  eaoh  group  prototype.  Negative 
distanoes  lndioate  that  the  individual  is  on  the  wrong  side  of  the 
deoision  surfaoe  and  therefore  misolassified. 
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Table  18 


Separation  of  UPTs  from  Other  Groups 
Oaaed  on  a  Minimum-Distance  Classifier 


Split-Plane  Maneuvers 
U1  through  U 1 7  are  individual  UPTs 


Rating  Snores 


Diatanoes  from  Distances  from 

Deeision  Surface  Group  Prototypes 


IP 

GP 

IWSO 

IP 

GP 

IWSO 

UPT 

U1 

14.95 

-0.45 

2.55 

66.51 

56.73 

59.20 

57.04 

U2 

17.07 

-3.03 

7.52 

66.44 

53.37 

61 .81 

55.48 

U3 

11.21 

11.70 

19.37 

45.58 

45.68 

55.80 

34.64 

UK 

19.47 

26.11 

31.13 

56.04 

59.96 

68.51 

40.20 

U5 

25.03 

35.90 

39.89 

66.31 

71.82 

79.87 

49.37 

U6 

21.72 

29.88 

36.71 

75.16 

78.82 

87.04 

62.84 

U7 

10.37 

-4.57 

-0.98 

58.09 

4  7,08 

49.66 

50.63 

U8 

1H. 76 

8.92 

15.01 

52.97 

48.23 

55.98 

40.63 

U9 

27.37 

36.00 

42.89 

72.52 

76.45 

85.76 

55.83 

U10 

16.28 

-0.54 

4.99 

62.92 

51.42 

56.37 

51. B2 

U11 

23.58 

37.16 

43.73 

68.84 

75.56 

84.94 

53.79 

U12 

21.06 

23.97 

28.12 

56.58 

50.05 

65.81 

39.40 

U 1 3 

27.13 

28.26 

32.67 

75.48 

75.60 

82.47 

59.78 

U1 4 

2D. 64 

35.48 

39.20 

62.25 

68.08 

76.28 

44.11 

U 1 5 

15.21 

0.86 

9.46 

69.80 

61.21 

67.95 

60.68 

U 16 

11.41 

16.99 

23.14 

60.07 

63.27 

70.73 

52.10 

U 1 7 

31.46 

39-46 

44.62 

91.67 

94.51 

101 .74 

77.08 

Average 

from  group 

prototypes: 

65.13 

63.87 

71.17 

52.08 

Eaoh  row  of  the  table  represents  one  UPT.  Deoision  surfaces 
were  computed  for  separating  UPTs  from  the  three  remaining  g’cups. 
The  distance  from  eaoh  UPT  to  a  decision  surface  is  shown  along 
with  the  distance  from  eaoh  UPT  to  each  group  prototype.  Negative 
distances  indicate  that  the  individual  is  on  the  wrong  side  of  the 
deoision  surfaoe  and  therefore  misclassified. 
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Table  19 


Separation  of  UPTs  from  Other  Groups 
Based  on  a  Minimum-Distance  Classifier 


Split-Plar.e  Maneuvers 
U1  through  U 1 7  are  individual  UPTs 

Distances  in  Multidimensional  Space 


Distances  from  Distanoe^  from 

Decision  Surfaoe  Group  Prototypes 


IP 

OP 

IWS0 

IP 

GP 

IWSO 

UPT 

U1 

61.23 

57.37 

61.95 

181 . 

34 

174.20 

183.69 

140.96 

U2 

66.10 

52.62 

75.00 

181 . 

13 

168.42 

188.55 

136.96 

U3 

43.82 

35.19 

52.22 

160. 

65 

151.39 

167.88 

128.42 

U4 

62.19 

55.60 

63.03 

189. 

15 

180.86 

191.49 

150.  in 

U5 

63.05 

63.00 

57.49 

182. 

09 

176.80 

180.63 

140.55 

U6 

38.62 

26.44 

37.63 

168. 

59 

158.24 

169.23 

142.17 

U7 

36.98 

23.21 

24.98 

165. 

60 

154.27 

158.60 

139.86 

U8 

49.62 

46.08 

61.96 

170. 

14 

163.75 

179.65 

135.64 

U9 

57.85 

56.07 

63.02 

164. 

17 

157.78 

169.60 

121.06 

U10 

67.88 

42.93 

60.85 

177. 

25 

157.56 

174.96 

130.33 

U11 

64,6? 

44.46 

61.10 

179. 

92 

163.57 

179.77 

136.51 

U12 

43.15 

47.75 

45.83 

171. 

88 

170.55 

175.02 

142.72 

U1 3 

57.82 

48.31 

52.08 

172. 

08 

161.68 

170.24 

131.61 

U14 

53.54 

60.08 

69.74 

166. 

32 

165.07 

178.59 

127.59 

U 1 5 

50.68 

38.96 

64.92 

169. 

43 

158.27 

180.20 

133.92 

U16 

34.:: 

21.5’ 

37.45 

165. 

15 

154.59 

168.38 

141.30 

U 1 7 

51.97 

56.49 

62.43 

172. 

49 

170.36 

100.79 

136.77 

Average 

from  group 

prototypes: 

172. 

79 

163.96 

176.31 

Eaoh  row  of  the  table  represents  one  UPT.  Decision  surfaoes 
were  aomputed  for  separating  UPTs  from  the  three  remaining  groups. 
The  diatartoe  from  eaoh  UPT  to  a  decision  surface  is  shown  along 
with  the  distance  from  eaoh  UPT  to  eaoh  group  prototype.  Negative 
distanoea  indioate  that  the  individual  is  on  the  wrong  side  of  the 
deolslon  surface  and  therefore  raisoiessified. 
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Table  20 


Separation  of  IPs  from  Other  Groups 
Baaed  on  a  Minimum-Distance  Classifier 

Low-Angle  Strafe 

II  through  16  are  individual  IPs 
Rating  Scores 


Distanoes  from 

Distanoes  from 

Deoision 

Surfaoe 

Group 

Prototypes 

IWSO 

UPT 

11 

IWSO 

UPT 

11 

26.30 

20.51 

43.13 

56.92 

58.81 

12 

3.24 

14.30 

32.23 

34.75 

46.41 

13 

20.65 

10.73 

54.67 

63.72 

61.84 

14 

8.41 

17.43 

28.48 

35.32 

46.58 

15 

1.32 

21.32 

41.67 

42.49 

58.30 

16 

17.62 

32.61 

56.23 

63.85 

72.52 

Average  from  group  prototypes: 

42.74 

49.51 

57.41 

Distances  in  Multidimensional  Spaoe 


Distanoes  from 

Distanoes  : 

from 

Deoision 

Surfaoe 

Group 

1  Prototypes 

IWSO 

UPT 

IP 

IWSO 

UPT 

11 

54.02 

70.32 

103.18 

137.45 

166.25 

12 

36.05 

62.14 

106.81 

130.04 

162.56 

13 

15.82 

33.90 

119.01 

128.75 

149.51 

14 

35.29 

70.55 

97.38 

121 .94 

162.89 

15 

57.26 

69.51 

134.06 

163.44 

186.46 

16 

30.45 

56.07 

119.50 

137.63 

166.82 

Average  from  group  prototypes: 

113.32 

136.54 

165.75 

Eaoh  row  of  the  table  represents  one  IP.  Deoision  surfaces 
were  computed  for  separating  IPs  from  the  three  remaining  groups. 
The  distance  from  eaoh  IP  to  a  deoision  surfaoe  is  shown  along 
with  the  distance  from  eaoh  IP  to  eaoh  group  prototype.  Negative 
distances  indioate  that  the  individual  is  on  the  wrong  side  of  the 
deoision  surfaoe  and  therefore  misolassif led. 
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Table  21 


Separation  of  IWSOa  from  Other  Groups 
Based  on  a  Minimum-Distance  Classifier 

Low-Angle  Strafe 

W1  through  W7  are  individual  IWSOa 
Rating  Soores 


Distances  from 

Leoision 

Surfaoe 

IP 

UPT 

W1 

22.16 

36.12 

W2 

15.65 

19.92 

W3 

9.81 

25.51 

W9 

12.39 

17.99 

W5 

7.51 

-17.61 

W6 

7.99 

15.69 

W7 

15.81 

39.25 

BTstanoea  from 
Group  Prototypes 


IP 

IWSO 

UPT 

60.20 

99.79 

71.28 

95.91 

35.36 

97.86 

92.09 

35.97 

55.68 

59.68 

98.95 

60.38 

75.98 

72.85 

63.53 

39.53 

39.26 

97.99 

53.30 

99.95 

67.03 

Average  from  group  prototypes:  02.95  95.87  59.11 


Diatanoes  in  Multidimensional  Space 


Diatanoea  from  Distances  from 

Decision  Surfaoe  Group  Prototypes 


IP 

UPT 

IP 

IWSO 

UPT 

W1 

27.63 

58.99 

129.55 

112.10 

155.23 

W2 

56.30 

99.06 

195.20 

111.75 

195.51 

W3 

31.99 

68.82 

129.03 

108.83 

159.91 

W9 

31.98 

90.32 

139.63 

120.89 

150.21 

W5 

69.28 

31.11 

193.33 

163.71 

1 8 1 .98 

W6 

19.65 

93.06 

129.96 

112.32 

195.27 

W7 

30.80 

59.11 

132.90 

113.27 

156.97 

Average  from  group  prototypes: 

192.01 

120.91 

156.23 

Eaoh  row 

of  the  table  represents 

one  IWSO. 

Decision 

surfaces 

were  computed  for  separating  IWSOSs  from  the  three  remaining  groups. 
The  distance  from  eaoh  IWSO  to  a  decision  surfaoe  is  shown  along 
with  the  distance  from  eaoh  IWSO  to  eooh  group  prototype.  Negative 
diatanoes  indiaate  that  the  individual  is  on  the  wrong  side  of  the 
deoision  surfaoe  and  therefore  miaolassified. 
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Table  22 


Separation  of  UPTa  from  Other  Groups 
Based  on  a  Minimum-Distanoe  Clasaifier 

Low-Angle  Strafe 

U1  through  U 1 6  are  individual  UPTs 
Rating  Soores 


Diatanoei 

i  from 

Diatanoea 

from 

Deoision  Surfaoe 

Qroup  Prototypes 

IP 

IWSO 

IP 

IWSO 

UPT 

U1 

7.36 

-1.72 

51.93 

44.71 

46.08 

U2 

11.60 

4.56 

49.68 

43.51 

39.55 

U3 

29.15 

35.96 

65.94 

68.36 

45.57 

U4 

27.22 

27.41 

67.99 

66.93 

50.01 

U5 

16.95 

4.60 

82.66 

76.44 

74.24 

U6 

7.05 

-4.04 

54.49 

46.13 

49.19 

U7 

12.70 

2.31 

58.80 

51.32 

49.67 

U0 

31 .5*4 

40.76 

77.06 

80.14 

58.99 

U9 

7.28 

-2.56 

54.49 

47.09 

49.01 

U10 

28.63 

44.02 

78.89 

84.68 

63.19 

U11 

20.4*1 

22.39 

55.11 

55.33 

38.00 

U12 

11.19 

7.93 

67.47 

65.21 

60.66 

U13 

18.53 

19.68 

57.72 

57.51 

43.44 

U14 

15.22 

8.03 

69.84 

65.35 

60.75 

U15 

27.89 

39.58 

73.30 

77.83 

56.57 

U16 

39.01 

39.88 

91.78 

90.89 

73.37 

Average  from  group  prototypes: 

66.07 

63.84 

Baoh  row  of  the  table  repreaenta  one  UPT.  Deoision  surfaoes 
were  oomputed  for  separating  UPTa  from  the  three  remaining  groups. 
The  distanoe  from  eaoh  UPT  to  a  deoision  surfaoe  la  shown  along 
with  the  diatanoe  from  eaoh  UPT  to  eaoh  group  prototype.  Negative 
diatanoea  Indicate  that  the  individual  is  on  the  wrong  aide  of  the 
deoision  surface  and  therefore  miaolasslfled. 
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Table  23 


Separation  of  UPTs  from  Other  Croupe 
Baaed  on  a  Minimum-Distance  Classifier 


Low-Angle  Strafe 

U1  through  U 1 6  are  individual  UPTs 
Distances  in  Multidimensional  Space 


Lstanoea  from 


Distances  from 


Deoiaion  Surfaoe 


Group  Prototypes 


IE 

IWSO 

IP 

IWSO 

UPT 

U1 

NN.98 

32.21 

179.23 

166.14 

145. 7B 

U2 

61.68 

49.43 

184.30 

169.72 

138.07 

U3 

79.67 

66.81 

190.52 

173.83 

130.57 

UN 

62.93 

56.  42 

194.63 

183.83 

150.57 

U5 

58.39 

37.43 

184.78 

165.56 

141.54 

U6 

69.89 

62.82 

188.00 

175.60 

135.84 

U7 

66.20 

59.78 

193.22 

181.99 

146.07 

U8 

68.56 

53.61 

173.44 

155.19 

116.25 

U9 

63.69 

52.99 

180.27 

165.98 

130.78 

U10 

58.86 

N9.00 

168.78 

154.67 

119.43 

U11 

67.67 

55.88 

201.38 

187.66 

155.57 

U12 

19.66 

15.18 

165.18 

159.77 

150.11 

U 1 3 

65.83 

62.59 

184.99 

175.07 

135.33 

U1N 

N9.09 

Ml  .74 

169.20 

158.09 

129.4B 

U 1 5 

55.38 

37.84 

183.86 

166.98 

142.91 

U 1 6 

7N.16 

54.82 

180.42 

159.49 

120.96 

Average  from  group  prototypes:  182.64  168.72  136.83 


Eaoh  row  of  the  table  represents  one  UPT.  Decision  surfaces 
were  oomputed  for  separating  UPTs  from  the  three  remaining  groups. 
The  distance  from  eaoh  UPT  to  a  decision  surfaoe  is  shown  along 
with  the  distanoe  from  eaoh  UPT  to  eaoh  group  prototype.  Negative 
distanoes  indioate  that  the  individual  is  on  the  wrong  side  of  the 
deoiaion  surfaoe  and  therefore  miaolnaaified. 
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As  mentioned  previously,  class  separation  depends  on  both  the 
distance  between  group  prototypes  and  the  closeness  with  which  members 
cluster  around  their  group  prototype.  Tables  IT)  through  .?!  give  a 
measure  of  class  clustering  in  the  form  or  the  average  distance  Trom 
the  class  prototype  to  each  class  member.  Shorter  distances  suggest, 
more  homogeneous  classes  with  greater  consistency  in  the  individual 
conceptual  structures.  The  average  distance  from  the  individual  to 
their  own  group  prototype  is  the  value  underlined  In  the  tables.  For 
both  maneuvers,  the  IPs  duster  most  tightly  while  UPTs  are  the  most 
variable.  This  la  reasonable  conaidering  that  IPs  follow  standard 
procedures  for  presenting  the  maneuvers  and  have  probably  developed 
similar  ways  of  thinking  about  them.  UPTs  on  the  other  hand  are  still 
learning  the  material  and  have  different  views  about  how  these 
oonoepta  are  related. 

All  of  the  classification  results  reported  so  far  have  cone  from 
applying  disoriminant  funotions  to  members  of  classes  from  which  the 
diaoriminant  funotions  ware  originally  derived.  Although  this  provides 
useful  Information  about  class  and  individual  differences,  it  is  not  a 
direct  test  of  the  ability  of  discriminant  functions  to  oategorize  new 
members  of  known  olaasea.  The  second  analysis  involved  generating  a 
disoriminant  function  on  the  basis  of  a  limited  training  set  from  two 
alasses  and  then  using  the  function  to  place  new  and  unknown  members 
into  one  of  the  two  olassas. 

The  results  of  the  classifications  are  given  in  Tables  24  and  25 
for  the  split-plane  maneuvers  and  Table  26  for  the  low-angle  strafe 
maneuver.  The  tables  give  the  total  number  of  individuals  for  whioh  a 
classification  was  attempted,  followed  by  the  percentage  of  thoso 
correctly  classified  and  the  probability  of  randomly  classifying  at 
least  this  number,  Sinae  100  different  randomly  chosen  training  nets 
were  used  for  each  training  set  size,  the  number  or  classifications 
attempted  is  always  100  times  the  number  of  remaining  members  in  the 
two  classes. 

The  classification  or  unknown  members  was  quite  successful.  Table 
24  shows  that  with  only  one  member  eaoh  from  the  IPs  and  CPs  on  which 
to  base  a  deolaion  surface,  798  out  of  1400  remaining  IPs  and  (IPs  worn 
classified  oorreotly.  Performance  was  evan  better  for  MDS  patterns 
where  949  of  the  1400  individuals  were  correctly  classified.  These 
results  are  more  Impressive  If  we  consider  that  two  of  the  seven  IPs 
resemble  OPs,  In  general,  olaasifioation  improves  as  the  distance 
between  olassea  in  the  pattern  apace  increases.  With  only  two  members 
eaoh  from  the  IPs  and  UPTs,  it  is  possible  to  classify  correctly  the 
remaining  20  members  95  peroent  of  the  time.  Table  26  shows  that  the 
classification  was  poorest  for  IPs  and  IWSOs  with  the  low  angle  strafe 
maneuver.  In  Table  14  the  distances  between  group  prototypes  show 
that  IPs  and  IWSOs  are  the  closest  pair  of  groups  for  both  ratings  and 
distances  in  MDS. 


Classification  generally  improves  also  an  the  size  of  the  train¬ 
ing  set  inoreases.  This  is  expected  since  the  discriminant  function 
is  derived  from  a  larger  more  representative  sample.  Exceptions 
sometimes  ooour  when  a  few  members  in  one  class  strongly  resemble 
members  of  the  other  class.  In  this  case,  when  the  training  set  size 
is  small,  the  chances  of  using  one  of  the  deviant,  class  members  to 
derive  the  disoriminant  funotion  is  also  small.  This  results  in 
generally  good  performance  over  the  100  trials,  But  as  the  training 
set  size  increases,  the  chances  or  baaing  the  discriminant  funotion  on 
one  of  the  deviant  members  inoreasns.  The  result  is  increased 
difficulty  in  classifying  tha  remaining  members.  Related  to  this  is  a 
faotor  that  influences  the  probability  measure.  Ao  the  number  of 
classifications  attempted  declines,  higher  percentages  of  oorreot 
olasaif iation  are  needed  to  maintain  previous  probability  levels, 

Disoriminant  funotions  baaed  upon  the  MDS  patterns  resulted  in 
better  classification  of  unknown  members  compared  to  the  rating 
patterns.  The  average  percentage  of  corroct  classifications  for  the 
MDS  patterns  was  significantly  greater  than  the  average  Tor  the 
ratings,  t(106)«5.95,  pCOOI.  As  mentioned  previously,  this  superior 
performance  oan  be  attributed  to  the  additional  structural  information 
supplied  by  MDS. 

It  is  important  to  note  that  the  pattern  recognition  analysis  has 
baan  performed  on  a  limited  number  of  Individuals.  Groups  have  ranged 
from  *4  to  17  members.  Traditional  applications  or  pattern  recognition 
often  uae  large  numbers  of  patterns  to  insure  that  decision  aurfaoes 
refleot  general  olaaa  distinctions.  The  suooess  seen  in  the  ourrent 
analysis  with  a  limited  number  of  patterns  suggests  considerable 
potential  for  futura  applications.  Decision  surfaces  based  on  large 
representative  groups  may  prove  to  be  even  more  effeotive  in  class¬ 
ifying  pilots  on  the  basis  of  conceptual  structures. 
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Table  24 


Classification  of  Groups  Members 
on  the  Basis  of  Limited  Training  Seta 

Split-Plane  Maneuvers 


Training 

Ratings 

Distances 

in  MDS 

Set  Size 
IPs  QPs 

•Classified 

JCorreot 

Prob* 

SCorreot 

Prob* 

1 

1 

1400 

57 

.001 

68 

.001 

2 

2 

1200 

67 

.001 

77 

.001 

3 

3 

1000 

67 

.001 

82 

.001 

4 

4 

800 

68 

.001 

84 

.001 

5 

5 

600 

62 

.001 

87 

.001 

6 

6 

400 

52 

ns 

87 

.001 

7 

7 

200 

£1 

ns 

64 

.001 

Total 

5000 

63 

.001 

79 

.001 

Training 

Ratings 

Distanoes 

in  MDS 

Set  Size 
IPs  IWSOs 

•Classified 

SCorreot 

Prob* 

^Correct 

Prob* 

1 

1 

900 

53 

.05 

56 

.001 

2 

2 

700 

57 

.001 

60 

.001 

3 

3 

500 

61 

.001 

63 

.001 

4 

4 

300 

2400 

n 

.001 

k 

.001 

Total 

50 

.001 

to 

.001 

Training 

Ratings 

Distanoes 

in  MDS 

Set 

IPs 

Size 

UPTs 

•Classified 

HCorrect 

Prob* 

SCorrect 

Prob* 

1 

1 

2200 

65 

,001 

79 

.001 

2 

2 

2000 

77 

.001 

95 

.001 

3 

3 

1800 

78 

.001 

98 

.001 

4 

4 

1600 

83 

.001 

100 

.001 

5 

5 

1400 

89 

.001 

100 

.001 

6 

6 

1200 

91 

.00! 

100 

.001 

7 

7 

1000 

IS 

.00! 

100 

.001 

Total 

11200 

80 

.oof 

94 

.001 

*  Probability  of  randomly  classifying  at  least  the  number  of 
individuals  oorreotly  classified  (namot  significant) . 
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Table  25 


Clarification  of  Ore  ups  Member.' 
on  tha  Baaia  of  Limited  alning  Sets 

Split. -Plane  Maneuvers 


Training 

Rati 

ngn 

Distances 

irm — 

Set  Slae 
oFa  fwSOa 

♦Classified 

SCorreot 

Prob* 

SCorreot 

Prob* 

1 

1 

1100 

58 

.001 

62 

.001 

2 

2 

900 

64 

.001 

71 

.001 

3 

3 

700 

65 

.001 

76 

.001 

4 

4 

£00 

1° 

.001 

Ik 

.001 

Total 

3200 

63 

.001 

70 

.001 

Training 

Ratings 

Distances 

in  MDS 

§&  aig* 

0P»  UPTs 

♦Classified 

HCorreot 

Prob* 

^Correct 

Prob* 

1 

1 

2400 

62 

.001 

79 

.001 

2 

2 

2200 

70 

.001 

89 

.001 

3 

3 

2000 

74 

.001 

93 

.001 

4 

4 

1800 

76 

.001 

95 

.001 

5 

5 

1600 

76 

.001 

95 

.001 

6 

6 

1400 

76 

.oo1 

95 

.001 

7 

7 

1200 

73 

,001 

95 

.001 

8 

8 

1000 

73 

.001 

96 

,001 

9  9 

Total 

800 

1W0 

12 

72 

.001 

.001 

2k 

92 

.001 

TooT 

Training 

Ratings 

Dlstanoas 

in  MDS 

Set  SUe 
iflSOa OPTa 

♦Classified 

ICorrsot 

Prob* 

flCorraot 

Prob* 

1 

1 

1900 

68 

.001 

76 

.001 

2 

2 

1700 

81 

.001 

93 

.001 

3 

3 

1500 

81 

.001 

94 

.001 

4 

4 

1300 

TO 

28 

.001 

95 

.001 

Total 

n 

7o5T 

n 

.001 

•  Probability  of  randomly  classifying  at  laaat  the  number  of 
individuals  oorraotly  olaaaified  (nsmot  significant). 
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Table  26 


Classification  of  Croups  Members 
on  the  Basis  of  Limited  Training  Sets 

Low-Angle  Strafe 


Training  Ratings  Distances  in  MDS 


Set 

Size 

(•Classified 

iCorreot 

Proh* 

iCorreot 

Prob* 

IPS 

IWSOs 

1 

1 

1100 

49 

ns 

56 

.001 

2 

2 

900 

43 

ns 

54 

.01 

3 

3 

700 

45 

ns 

53 

ns 

4 

4 

500 

47 

ns 

55 

.05 

5 

5 

300 

43 

ns 

53 

ns 

6 

6 

100 

56 

ns 

56 

ns 

Total 

3555 

M 

ns 

ft 

7557 

Training 

Ratings 

Distanoea 

in  MDS 

IPs 

Size 

UPTs 

•(Classified 

iCorreot 

Prob* 

mmmammmm § 

iCorreot 

Prob8 

1 

1 

2000 

59 

.001 

84 

.001 

2 

2 

1800 

72 

.001 

96 

.001 

3 

3 

1600 

73 

.001 

96 

.001 

4 

4 

1400 

81 

.001 

97 

.001 

5 

5 

1200 

79 

.001 

97 

.001 

6 

6 

1000 

80 

.001 

25 

.001 

Total 

9000 

73 

.001 

94 

.001 

Training 

Ratings 

Distances 

in  MDS 

Set 

Size 

(•Classified 

iCorreot 

Prob* 

iCorreot 

Prob* 

IWSOs 

1 

UPTs 

1 

2100 

58 

.001 

69 

.001 

2 

2 

1900 

62 

.001 

88 

.001 

3 

3 

1700 

65 

.001 

91 

.001 

4 

4 

1500 

67 

.001 

92 

.001 

5 

5 

1300 

67 

,001 

93 

.001 

6 

6 

1100 

60 

.001 

93 

.001 

7 

7 

900 

54 

,001 

87 

.001 

Total 

10500 

6l 

.001 

57 

",oof 

•  Probability  of  randomly  classifying  at  least  the  number  of 
individuals  oorreotly  classified  (nsnnot  significant). 
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The  final  analysis  on  tho  qualitative  differences  between  groups 
examined  the  distanoa  between  eaoh  pair  of  oonoepts  in  a  group's 
oonoeptual  struoture.  The  result  was  a  list  of  the  concept  pairs  that 
were  most  different  in  the  oonoeptual  structures  for  two  groups.  The 
list  is  separated  into  oonoept  pairs  that  one  group  views  as  more 
related  than  the  other  group.  Later  in  this  report,  a  detailed  anal¬ 
ysis  of  IP  and  UPT  oonoeptual  structures  is  given  based  on  a  general 
weighted  network.  To  allow  a  comparison  of  the  upcoming  analysis  with 
the  results  of  the  pattern  recognition  techniques,  Table  27  shows 
those  split-plane  oonoepts  that  lead  to  the  most  disagreement  between 
IPs  and  UPTs. 

The  major  goal  of  this  phase  of  tho  project  has  been  to  demon¬ 
strate  both  the  feasibility  and  utility  of  applying  the  techniques  and 
prinoiplea  of  pattern  recognition  to  oonooptual  structures  of  oritioal 
flight  information.  The  general  finding  is  that  pattern  recognition 
teohniquea  sppasr  to  be  sensitive  enough  to  detect  subtle  differences 
between  Loth  groups  and  individuals.  In  addition,  these  differences 
often  aeem  to  have  "real-world"  significance.  Many  of  the  findings 
may  have  ralevanoe  to  aalaotion  and  training.  As  dlsousaed  later  in 
this  report,  a  baginning  pilot's  knowledge  of  split-plane  maneuvers 
bagins  as  a  fairly  disorganised  set  of  relations.  As  the  person 
undergoes  training  and  gains  experianoa  in  these  maneuvers  the 
oonoeptual  atruoturea  should  begin  to  evolve  into  structures  that  more 
aloaely  roaemble  thoae  of  experienced  pilots.  The  results  of  this 
part  of  tha  project  suggest  the  possibility  of  tapping  into  this 
devalopmental  prooaaa  and  olaaaifying  a  person  at  a  particular  stage 
of  oonoeptual  davalopment. 


-56- 


Table  27 


The  Most  Disorimlnable  Pairs  of  Concepts 
for  Distinguishing  IPs  from  UPTs 

Split-Plane  Maneuvers 


IPs  View  These  Pairs  as  More  Related  Than  UPTs 


csrww  -rmm - 

HIGH  VO  YO  -  BARREL  ROLL 
QUARTER  PLANE  -  RELATIVE  ENERGY 
QUARTER  PLANE  -  VERTICAL  MANEUVERING 
GUNS  -  LEAD  PURSUIT 
CUTOFF  -  SNAPSHOT 
POWER  SETTING  -  EXTENSION 
SWITCHOLOGY  -  WEAPONS  PARAMETERS 
RADIAL  G  -  VERTICAL  MANEUVERING 
HEAT  -  WEAPONS  PARAM1  IRS _ 


HIGH  YO  YO  -  CHARTER  PLANE 
HIGH  YO  YO  -  RADIAL  G 
BARREL  ROLL  -  ASPECT  ANGLE 
OUNS  -  CUTOFF 
ANGLE  OFF  -  6  O'CLOCK 
6  O'CLOCK  -  HEAT 
ACCELERATION  -  EXTENSION 
RADIAL  0  -  LIFT  VECTOR 
SMASH  -  EXTENSION 
SNAPSHOT  -  LEAD  PURSUIT 


UPTs  View  These  Pairs  as  More  Related  Than  IPs 


rartor-"irraH'T8,i8 - 

AIRSPEED  -  PURE  PURSUIT 

ANGLE  OFF  -  3-9  LINE 

CUTOFF  -  CORNER  VELOCITY 

6  O'CLOCK  -  SNAPSHOT 

POWER  SETTINQ  -  LEAD  PURSUIT 

SMASH  -  HEAT 

HEAT  -  LEAD  PURSUIT 

3-9  LINE  -  EXTENSION 

CORNER  VELQ CITY  -  LEAD  PURSUIT 


mrnzrrrm  'pursuit — 

ANOLE  OFF  -  CUTOFF 
CUTOFF  -  6  O'CLOCK 
CUTOFF  -  LAG  PURSUIT 
6  O'CLOCK  -  LEAD  PURSUIT 
ACCELERATION  -  LIFT  VECTOR 
SMASH  -  PURE  PURSUIT 
SNAPSHOT  -  LAG  PURSUIT 
LAO  PURSUIT  -  LEAD  PURSUIT 
PURE  PURSUIT  -  LEAD  PURSUIT 
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GENERAL  WEIGHTED  NETWORKS 

An  understanding  of  the  underlying  conceptual  structure  of  criti¬ 
cal  flight  information  can  be  advanced  by  determining  the  global 
relationships  among  the  concepts  as  we  have  done  using  conventional 
scaling  techniques  (e.g.,  MDS).  However,  representing  local  structure 
(i.e.,  detailed  relationships)  among  critical  flight  concepts  is 
quintessential  to  a  full  appreciation  of  fighter  pilots’  conceptual 
data  bases.  Theoretically,  a  general  weighted  network  can  accomplish 
this  goal. 

A  general  weighted  network  is  a  configuration  in  which  concepts 
are  depicted  by  nodes  and  relationships  are  depicted  by  links 
connecting  the  nodes.  The  links  are  assigned  a  value  or  weight  that 
reflects  the  strength  of  the  relationship  between  the  nodes.  The 
value  reflects  the  distance  from  one  node  to  another  along  that  link; 
the  shorter  the  link,  the  closer  the  nodes.  The  network  is  general  in 
that  constraints  are  not  placed  on  the  possible  relations  that  can  be 
represented.  For  example,  the  hierarchical  constraint  found  in 
cluster  analysis  is  not  placed  on  general  weighted  networks.  With 
this  constraint  removed,  the  representation  becomes  more  sensitive  to 
local  relations  other  than  hierarchical  ones  but  hierarchical 
relations  may  still  be  present  (Christofides,  1975;  Fillembaum  & 
Rapaport,  1971). 

Networks  have  formed  the  basis  of  research  in  a  number  of  areas 
of  cognitive  science.  Several  psychological  and  artificial  intell¬ 
igence  models  of  conceptual  structure  are  based  on  such  networks. 
Work  in  graph  theory  is  centrally  concerned  with  properties  c  ?  general 
networks.  While  important  theoretical  and  formal  work  nas  been 
conducted  on  these  structures,  no  methods  have  been  available  to 
produce  networks  from  empirically  obtained  measures  -of  psychological 
distance.  We  developed  a  network  algorithm  which  produces  general 
weighted  networks  (GWN)  to  apply  to  the  rating  data.  Recently  we 
discovered  another  algorithm  developed  by  Hutchinson  (1981).  Here  we 
will  discuss  our  algorithm,  GWN,  and  its  application. 

The  central  problem  in  constructing  a  network  from  psychological 
distance  data  is  to  determine  which  links  to  place  in  the  network. 
For  N  concepts,  the  possible  number  of  links  lies  between  N-1  links 
for  a  minimally  connected  network  (MCN)  and  (N  x  (N-1))/2  links  for 
all  possible  connections. 

CV.’M  'ecides  whether  to  add  a  link  v  '  r.c  existing  network  for  any 
two  concepts,  say,  OUNS  and  CUTOFF,  xn  the  following  way.  The  empir¬ 
ical  distance  between  the  two  concepts  is  compared  with  the  shortest 
chain  (sequence  of  links)  already  existing  in  the  network  connecting 
GUNS  and  CUTOFF,  Such  a  chain  might  be  GUNS-LF.AD  PURSUIT-CUTOFF,  or 
mo^  general'*  ;,  GUNS-X1-X2-  .  .  Xn-CUTOFF,  where  the  Xs  represent 
. .. .  e  ’ ■'*** '  ,ig  node3  of  the  chain.  GWN  assumes  that  for  a  person  to 
decide  on  the  relatedness  of  GUNS  and  niTOFF  search. is  directed  along 


-58- 


the  network  from  each  concept.  The  searches  will  intersect,  at  some 
concept  Xi,  along  the  shortest  chain  connecting  the  two  concepts.  The 
distance  before  this  intersection  occurs  is  taken  by  GWN  as  the  short- 
est  distance  between  the  concepts.  If  the  empirical  distance  is 
longer  than  the  shortest  distance  currently  in  the  network,  then  GWN 
does  not  add  a  link  since  the  new  link  would  be  redundant  with  the 
shortest  chain.  If  the  empirical  distance  is  shorter  than  the 
evaluated  distance  of  the  shortest  current  chain,  then  GWN  adds  a  link 
connecting  GUNS  and  CUTOFF  because  the  psychological  distance  is 
smaller  than  would  be  allowed  by  the  existing  network.  By  iterating 
this  procedure  starting  with  the  smallest  empirical  distance  and 
proceeding  with  all  distances  in  order  of  their  magnitude,  GWN  adds 
links  to  the  network  and  can  create  networks  of  varying  complexity. 

• 

MCN:  The  Skeletal  Structure.  For  any  set  of  N  concepts,  those 
concepts  can  be  formed  into  a  network  (more  specifically  a  tree)  with 
N-1  links.  In  such  a  network  or  tree,  each  concept  is  reachable  from 
every  other  concept.  Each  node,  not  yet  connected  to  the  network,  is 
connected  by  finding  the  shortest  link  between  it  and  an  element  of 
the  network.  This  solution  minimizes  the  average  distance  from  all 
concepts  Xi  to  all  concepts  Xj  for  N-1-  links.  The  MCN  ha3  a  special 
status  in  that  the  MCN  links  will  appear,  by  definition,  in  all 
connected  networks  regardless  of  how  elaborate.  In  a  senses  the  MCN 
represents  the  backbone  of  the  network  and  may  have  a  special  status 
in  the  conceptual  representation.  The  MCN  produced  by  GWN  turns  out 
to  be  the  shortest  spanning  subtree  proposed  by  Kruskal  (1956). 

MEN:  The  Integral  Connections.  The  MEN  is  the  minimal  network 
that  Includes  more  links  than  the  simple  tree  structure  of  the  MCN. 
The  MEN  allows  additional  Interconnections  among  concepts  that  produce 
the  mo3t  efficient  connections  with  a  minimal  number  of  links.  Unlike 
the  MCN,  the  MEN  does  not  have  a  logical  bound  on  the'  number  of  links 
that  can  be  found  in  the  network.  Also  unlike  the  MCN,  cycles  can  be 
introduced  into  the  representation.*  Cycles  are  of  particular  interest 
in  that  they  are  chains  that  begin  and  terminate  on  the  same  node. 

We  analyzed  the  psychological  distance  data  from  UPTs,  IPs,  GPs, 
and  IWSOs  for  split-plane  maneuvers  and  the  low-angle  strafe  maneuver 
using  GWN.  The  resulting  networks  for  IPs  and  UPTs  appear  in  Figures 
10  to  16.  The  nodes  in  the  networks  are  located  on  the  page  according 
to  the  two-dimensional  multidimensional  scaling  solutions  for  the 
IPs.  One  of  the  problems  of  representing  the  networks  is  arranging 
the  nodes.  Using  the  MDS  solution  solves  that  problem  and l"*h as  the 
advantage  of  depicting  both  dimensional  information  and  network 
.information  in  tbs  same  representation,  ^he  MDS  solution  for  the  IPs 
is  used  for  ali  networks  To  <^^<ait?te  comparisons  between  IP  and  UPT 
networks.  In  the  figures',  »-he  noNs  are  represented  by  solid  lines  and 
the  dotted  lines  represent  the  additional  links  added  in  the  MEN 
solutions. 
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Figure  10.  Minimal  Connected  Network  by  GWW  of  Split  Plane  Concepts  tor  Instructor  Pilots. 


Minima)  Elaborated  Network  by  GWK  of  Strafe  concepts  for  Instructor  Riots.  Solid 


The  network  derived  from  the  student  data  is  considerably  more 
complex  than  the  IP  network.  The  elaborated  student  network  has  51 
links  compared  to  40  for  the  instructor  network.  Apparently,  the 
instructors  have  a  better  organized  struoture  for  these  oonoepts;  at 
least  it  is  more  elegant.  Surprisingly ,  the  OP  network  was  even  more 
complex  than  the  UPT  network.  Perhaps  the  necessity  of  ooraraunioat ing 
the  oonoepts  to  students  contributes  to  the  simplicity  of  the  instruct¬ 
ors'  organization. 

Ne  will  analyze,  in  depth,  the  network  of  split-plane  concepts 
for  the  IPs  and  then  will  make  comparisons  across  networks.  The  IP 
network  was  chosen  for  several  reasons:  a)  The  network  oontains 
several  theoretical  highlights  of  networks,  b)  The  network  is  the 
representation  of  an  expert  and  is  therefore  of  Hraotioal  importance, 
o)  it  is  similar  to  the  other  group  of  professional  pilots  (i.e.  CPs), 
d)  the  IPs  have  direat  communication  with  the  UPTs,  and  e)  the  IPs 
produced  the  most  elegant  struoture,  whioh  will  simplify  discussion 
considerably. 

Split  Plane  Network  for  Instructor  Pilots.  We  analyzed  the 
empirical  distanoe  data  for  the  30  SPLIT  PLANE  oonoepts  usind  OWN. 
OWN  produoed  the  MCN  shown  in  Figure  10  for  the  instructor  pilots. 
The  network  is  oomposed  of  29  links  with  an  average  link  length  of 
10,4  and  a  standard  deviation  of  7.  The  shortest  link  is  between  GUNS 
and  SNAPSHOT  (1  unit)  and  the  longest  links  are  LAO  ROLL-ASPECT  ANQLE 
and  HEAT-PURE  PURSUIT  (26  units).  A  great  deal  of  struoturo  is 
apparent  even  in  the  skeletal  configuration  of  the  MCN.  The  MCN  rot 
these  data  is  far  from  the  simple  structures  in  whioh  one  concept 
links  to  the  remaining  29  oonoepts  or  in  whioh  eaoh  conoept  links  to 
two  other  oonoepts  forming  a  straight-line  ohain.  Rather  there  are 
several  concepts  that  link  to  multiple  oonoepts  (GUNS,  HEAT,  ASPECT 
ANGLE,  VERTICAL  MANEUVERING,  ACCELERATION,  SMASH,  AIRSPEED). 

When  OWN  elaborates  the  MCN  to  form  the  MEN,  11  additional  links 
are  added,  making  a  total  of  40  links,  with  an  average  link  length  of 
14.5,  standard  deviation  of  10.5,  and  maximum  link  length  of  44  units 
(AIRSPEED-PURE  PURSUIT).  Figure  11  shows  the  MEN  solution  with  the 
MCN  links  as  solid  lines  and  the  additional  11  links  as  dotted  lines. 
The  additional  links  seem  to  integrate  the  overall  network  by  connect¬ 
ing  oonoepts  olosely  allied  with  flight  to  oonoepts  olosely  allied 
with  weapons.  Some  of  these  oonneotions  are  indirect  (e.g.,  QUARTER 
PLANE  to  LAG  PURSUIT)  while  others  are  quite  direct  (e.g.,  0  L0ADIN0 
to  WEAPONS  PARAMETERS).  The  three  new  links  from  QUARTER  PLANE 
oertalnly  help  interoonneot  the  overall  network  and,  in  addition,  make 
QUARTER  PLANE  a  central  oonoept  in  tho  network.  Note  also  that  only 
four  oonoepts  are  terminal  (i.e.,  conneot  to  only  one  other  oonoept): 
HI  YO  YO,  LO  YO  YO,  6  O'CLOCK,  and  CORNER  VELOCITY. 
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Another  very  important  form  of  integration  produced  by  the 
additional  links  of  the  MEN  is  the  establishment  of  oyoles.  Cyoles 
are  ohains  of  links  through  the  network  that  allow  a  return  to  the 
starting  point  without  baoktraoking  (i.e.,  without  returning  over  the 
same  link).  Reoall  that  with  the  MCN  configuration  of  29  links, 
oyoles  were  not  possible.  The  addition  of  the  11  links  in  the  MEN 
produced  nine  minimum  distance  cycles.  Cycles  cnn  be  round  for  each 
nonterminal  node  in  the  network.  The  terminal  nodes  obviously  cannot 
be  in  cycles  since  there  is  only  one  entry  point  to  the  concept. 

Viewing  the  nine  oyoles  in  Figure  12  gives  a  better  pioture  of 
the  structure  present  in  the  MEN.  In  principle  there  oould  be  26 
minimum  oyoles  In  the  MEN,  one  for  each  non-terminal  oonoept.  However, 
only  nine  unique  minimum  oyoles  are  present  in  the  MEN  indioatlng  that 
several  oonoepts  share  the  same  cycle.  The  shortest  oyole  is  only  20 
units  long  and  is  used  by  five  oonoepts  as  the  minimum  oyole.  The 
longest  oyole  is  127  units  long  and  is  the  minimum  oyole  for  only  one 
oonoept  (i.e.,  PURE  PURSUIT). 

Members  of  the  same  oyole  may  have  a  psychological  relationship 
among  thamsalvaa  that  ia  not  found  between  oonoepts  from  different 
oyolea.  For  example,  the  three  shortest  cycles  are  of  particular 
interest,  not  only  baoause  they  are  the  most  oompact,  but  also  beoause 
aaoh  member  of  the  oyole  uses  that  cycle  as  the  shortest  route  through 
the  network  and  baok.  These  oyoles  appear  to  form  meaningful 
organizational  units! 

(AIRSPEED-SMA3H-P0WER  SETTINO-EXTENSION-ACCELERATIQN ) 
(QUNS-SVUTCHOLOQY-HEAT- WEAPONS  PARAMETERS) 

(QUARTER  PLANE-VERTICAL  MANEUVERING- 

BARREL  ROLL-ASPECT  ANGLE-3 /9  LINE) 

Another  means  of  identifying  substructures  within  the  MEN  is  the 
formation  of  oonoeptual  assemblies.  Conceptual  assemblies  art  formed 
by  finding  the  smallest  number  of  oonoepts  that  dominate  all  the  other 
oonoepts  in  the  network.  In  other  words,  from  this  set  of  dominating 
oonoepts  (DCs)  all  other  oonoepts  oan  be  reaohed  by  traversing  one 
link.  Suah  a  sat  ia  referred  to  in  graph  theory  as  externally 
stable.  Tha  problem  of  finding  auoh  a  set  is  analogous  to  determining 
the  number  and  locations  of,  ssy,  army  posts  to  guard  oitiaa  (nodes) 
oonnaotad  by  major  highways  (links)  or  hospitals  to  sarvlos 
naighborhoods  oonnaotad  by  streets.  Often  several  auoh  sets  satisfy 
tha  requirements  of  an  externally  stable  set.  In  forming  assemblies, 
wa  plaoed  the  additional  restriction  that  the  set  be  an  close  as 
possible  to  internally  stable.  An  Internally  stable  set  is  s  set  of 
oonoepts  with  no  shared  links.  Thus,  we  attempted  to  rind  the  set  of 
DCs  that  had  a  link  with  every  other  concept  in  the  network  but  with 
no  links  to  other  DCs. 
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The  DCs  for  this  network  were  GUNS,  HEAT,  QUARTER  PLANE,  ACCEL¬ 
ERATION,  SMASH,  CORNER  VELOCITY,  RADIAL  G,  CUTOFF,  and  ASPECT  ANGLE. 
Assemblies  are  composed  of  the  DC  and  all  concepts  that  it  dominates. 
For  example,  the  GUNS  assembly  contains  GUNS,  SWITCHOLOGY,  WEAPONS 
PARAMETERS,  SNAPSHOT,  and  LEAD  PURSUIT.  Just  as  cycles  partition  the 
MEN  into  a  small  number  of  comprehensible  subnetworks  (i.e.,  nine)  so 
do  assemblies  break  down  the  large  amount  of  data  represented  in  the 
network  into  a  smaller  number  of  meaningful  units,  in  this  case  nine. 

The  network  of  split-plane  concepts  for  the  instructor  pilots 
highlights  a  number  of  local  relationships  that  are  not  apparent  in 
the  more  traditional  scaling  techniques  that  we  have  used.  The 
networks  themselves,  MCN  and  MEN,  reduce  a  huge  amount  of  data  in  the 
form  of  an  N  x  N  distance  matrix  to  a  much  smaller  set  of  data.  These 
networks  can  be  reduced  still  further  by  isolating  minimum  cycles  or 
DC  assemblies.  An  understanding  of  the  local  relationships  among 
concepts  or  among  cycles  or  assemblies  makes  it  possible  to  compare 
experienced  pilots  with  those  training  to  become  pilots.  With  other 
techniques,  say  MDS,  only  gross  global  comparisons  can  be  made  between 
the  groups.  With  the  network  representation,  and  with  the  pattern 
classifier  discussed  elsewhere  in  this  report,  individual  concepts  can 
be  examined  and  the  extent  to  which  the  organization  is  the  same  for 
experienced  and  novice  pilots  can  be  determined.  Further,  novice 
pilots  may  differ  from  experienced  pilots  either  because  the  novices 
are  missing  critical  links  for  a  particular  concept  (underdefined 
concepts)  or  because  they  have  additional  organizational  links  that 
the  more  experienced  pilots  do  not  have  (overdefined  concepts). 

Comparison  of  Novice  and  Expert  Pilots  using  GWN.  We  used  the 
GWN  algorithm  to  isolate  those  concepts  and  relations  that  distinguish 
expert  fighter  pilots  from  novices.  These  concepts  and  relations  fall 
into  two  categories:  (a)  those  that  experts  have  that  the  novices  do 
not  and  (b)  those  that  novices  have  that  experts  do  not.  The  under¬ 
graduate  pilots  served  as  the  novice  group,  and  the  instructor  and 
guard  pilots  were  the  experts. 

In  the  organization  of  any  set  of  concepts,  there  are  a  number  of 
idiosyncratic  relations  in  the  network  of  any  individual  person  or 
group  of  people.  In  order  to  avoid  these  idiosyncracies,  the  study 
focused  on  the  group  networks  for  expert  and  novice  fighter  pilots. 
We  were  interested  in  comparing  the  network  derived  from  the  UPT  data 
against  the  representation  of  critical  flight  information. 

Critical  flight  information  was  defined  as  the  segments  of  the 
MEN  that  were  shared  by  both  groups  of  experts, i.e. ,  the  links  con¬ 
tained  in  the  MEN  of  both  the  IPs  and  the  GPs.  If  the  student  network 
does  not  have  a  link  found  in  both  expert  networks,  the  student  may  be 
missing  a  piece  of  critical  flight  information.  Similarly,  if  the 
student  has  a  link  that  is  found  neither  in  the  IP  network  nor  in  the 
GP  network,  then  the  student  may  have  a  misconception  about  the 
relation  between  the  linked  concepts. 
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Critical  flight  network  and  assemblies.  GWN  found  the  MEN  for 
the  IPs  and  GPs.  The  intersection  of  these  MENs  formed  the  critical 
links  of  the  expert  pilots.  As  can  be  seen  in  Figure  16,  this  led  to 
a  disconnected  network  composed  of  three  connected  components  and 
three  concepts  that  did  not  have  any  critical  links.  These  three 
concepts  were  connected  to  the  IP  and  GP  networks  differently  and  thus 
do  not  produce  any  critical  links.  These  concepts  are  good  examples 
of  the  role  of  idiosyncratic  information  in  the  networks. 

For  each  connected  component,  the  DCs  and  accompanying  assemblies 
were  identified.  Note  that  there  were  no  critical  cycles  so  a  cycle 
analysis  was  not  done.  Table  28  lists  the  DCs  and  accompanying  assem¬ 
blies  for  each  connected  component.  If  a  DC  was  also  a  terminal  con¬ 
cept,  we  included  that  concept  within  the  larger  assembly  for  ease  of 
exposition.  The  DCs  for  the  critical  network  were:  GUNS,  CUTOFF,  VERT¬ 
ICAL  MANEUVERING,  AIRSPEED,  RELATIVE  ENERGY,  LAG  ROLL,  RADIAL  G,  3-9 
LINE.  The  Vertical  Maneuvering  Connected  Component  is  partitioned 
into  a  Guns  Subassembly,  Angle  Off  subassembly,  Vertical  Maneuvering 
subassembly,  and  an  Airspeed  subassembly.  In  the  Guns  Subassembly, 
CUTOFF  was  also  a  DC  as  was  RELATIVE  ENERGY  in  the  Airspeed  Sub- 
assembly.  The  Lag  Roll  Subassembly  forms  with  LAG  ROLL  as  the  center 
and  RADIAL  G  and  3-9  LINE  as  terminal  DCs.  The  Heat  Component  and 
Acceleration  Component  have  only  two  concepts  each  and  thus  are  not 
reducible  to  subassemblies.  There  is  considerable  similarity  between 
the  DCs  and  assemblies  found  for  the  IPs  earlier  and  the  critical  DCs 
and  assemblies  found  in  this  analysis. 

The  critical  links  in  the  assemblies  are  underlined.  The  three 
columns  next  to  each  link  indicate  whether  the  MEN  for  each  of  the 
three  groups  of  pilots  contained  that  link.  For  example,  within  the 
Guns  Subassembly  for  the  concept  LEAD  PURSUIT  we  find  three  links. 
Two  of  the  links  are  critical  and  shared  by  the  IPs  and  GPs  (GUNS  and 
CUTOFF),  and  one  of  the  links  is  noncritical  and  held  by  only  the 
students  (OVERTAKE).  A  perusal  of  the  table  shows  that  some  links  are 
possessed  by  only  one  group,  some  links  are  shared  by  UPTs  and  IPs  or 
by  UPTs  and  GPs  or  by  IPs  and  GPs  (critical),  and  some  links  are 
shared  by  all  (critical  links  held  by  the  students). 


’'"  ™ 1 1  "WHBpWF1'"' 


KEY: 


Table  28 

Assemblies  for  the  Split-Plane  Concepts 


The  table  (which  appears  on  the  following  several  pages)  depicts 
links  oonneoted  to  oonoepts  a'ooor ding  to  OWN  solutions  for  student 
pilots,  Instructor  pilots,  and  national  guard  pilots.  Minimal  elab¬ 
orated  networks  were  used  for  the  students  and  guards,  while  an  elab¬ 
orated  network  with  51  links  was  used  for  the  instructors  in  order  to 
equate  number  of  direot  oonoept-to-oonoept  links. 

In  the  left-hand  oolumn,  elements  of  the  network  are  listed,  The 
next  oolumn  oontains  the  elements  that  are  linked  to  the  elements 
listed  iu  the  first  oolumn.  The  X's  indioats  which  pilot  groups  have 
that  link. 

Underlined  oonoepts  and  corresponding  X's  represent  critical 
links.  A  oritioel  link  is  one  thit  is  in  the  network  of  both  the 
instructors  and  guards. 

ASSEMBLIES  used  oritioel  links  and  were  formed  in  the  following 

ways: 


1)  A  disconnected  network  was  formed  from  the  oritioel  links  and 
nodes. 

2)  Each  oonneoted  oomponent  wee  labelled  by  its  graph  theoretic 
median  (or  if  no  relative  median  oould  be  uniquely  determined,  one 
was  ohoson).  This  led  to  four  oonneoted  components  (l.e.,  VERTICAL 
MANEUVERING,  LAO  ROLL,  HEAT,  ACCELERATION)  end  three  Independent 
oonoepts  (i.e.,  BARREL  ROLL,  OVERTAKE,  LOW  YO  YO). 

3)  The  minimal  externally  stable  seta  of  maximally  dominating 
oonoopta  (MDC)  were  determined  for  eaoh  oonneoted  oomponent  and  the 
set  that  was  most  Internally  stable  was  used  to  form  subassemblies  of 
the  oonneoted  components. 


(Table  28  continued) 

VERTICAL  MANEUVERING  CONNECTED  COMPONENT 


UPTs 

IPs 

GPs 

GUNS  (MDC) 

SWITCHOLOGY 

X 

X 

X 

SNAPSHOT 

X 

X 

X 

WEAPONS  PARAMETERS 

X 

X 

X 

LEAD  PURSUIT 

X 

X 

6  O'CLOCK 

X 

- 

SWITCHOLOGY 

GUNS 

X 

X 

X 

HEAT 

X 

SNAPSHOT 

GUNS 

X 

X 

X 

(shared  by 

ANGLE  OFF 

X 

X 

angle  off 

3-9  LINE 

X 

subassembly) 

BARREL  ROLL 

X 

ASPECT  ANGLE 

X 

G  LOADING 

X 

OVERTAKE 

X 

WEAPONS  PARAMETERS 

GUNS 

X 

X 

X 

EXTENSION 

X 

ASPECT  ANGLE 

X 

X 

LAO  ROLL 

X 

ANGLE  OF!- 

X 

X 

RADIAL  G 

X 

HEAT 

X 

X 

G  LOADING 

X 

6  O'CLOCK 

X 

HEAT  ASSEMBLY 

X 

X 

X 

LEAD  PURSUIT 

GUNS 

X 

X 

CUTOFF 

X 

X 

OVERTAKE 

X 

X 

CUTOFF  (MDC) 

LEAD  PUHSUIT 

X 

X 

3-9  LINE 

X 

ASPECT  ANGLE 

X 

X 

OVERTAKE 

X 

X 

RELATIVE  ENERGY 

X 

ANGLE  OFF 

X 

LO  YO  YO 

X 
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(Table  28  continued) 


VERTICAL  MANEUVERING  SUBASSEMBLY 


UPTs  IPs  GPs 


VERTICAL 

MANEUVERING  (MDC) 
([-■  titled  by 
airspeed 
subassembly) 


HI  YO  YO 
LIFT  VECTOR 
QUA ft TEH  PLANS 
AIRSPEED 
BARREL  ROLL 
RELATIVE  ENERGY 
3-9  LINE 
LAG  ROLL 

LAG  ROLL  hSSEMBLY 


X 

X 


X 

X 


X 


HI  YO  YG 


VERTICAL  MANEUVERING 
LO  YO  YO 
3-9  LINE 
ASPECT  ANGLE 
QUARTER  PLANE 
BARREL  ROLL 


XXX 

X 

X 

x  X 

x 

X 


LIFT  VECTOR 


VERTICAL  MANEUVERING  X 
QUARTER  PLANE  X 
G  LOADING  X 
RADIAL  G 

LAG  ROLL  ASSEMBLY  (G)  X 


QUARTER  PLANE 
(shared  by 
angle  off 
subassembly) 


VERTICAL  MANEUVERING 
ANGLE  OFF  X 

3-9  LINE  X 

LIFT  VECTOR  X 

EXTENSION  X 

ASPECT  ANGLE  X 

LAG  PURSUIT  X 

LO  TO  YO  x 

HI  YO  YO 
RADIAL  G 
6  O'CLOCK 

LAG  ROLL  ASSEMBLY  (3-9)  X 


X 

X 

X 


X 

X 

X 

X 


X 

X 


X 


X 

X 


X|X|X|X|X  x  X| 


1 


1 


1 


I 


I 


I 


(Table  28  continued) 

UPTs  IPa  GPa 


AIRSPEED  SUBASSEMBLY 

AIRSPEED  (MDC) 

(shared  by 

vertloal 

maneuver 

subassembly) 


SMASH 

CORNER  VELOCITY 
PURE  PURSUIT 
VERTICAL  MANEUVERING 
POWER  SETTING 
EXTENSION 


OVERTAKE 

ACCELERATION 


LO  YO  YO 

ACCELERATION  ASSEMBLY 


SMASH  AIRSPEED 

RELATIVE  ENERGY 
HEAT  — 
ACCELERATION 
OVERTAKE 
POWER  SETTING 
EXTENSION 
0  LOADING 

ACCELERATION  ASSEMBLY 


« 


1 


CORNER  VELOCITY  AIRSPEED 

Q  LOADING 

PURE  PURSUIT  AIRSPEED 

6  O'CLOCK 
HEAT 

ACCELERATION 
RELATIVE  ENERGY 
HEAT  ASSEMBLY 


XXX 
X  X 


X 

X 


X 


X 

X 

X 

X 

X 


X 

X 


X 


,1 

a 

t 


■i! 

‘l 

i 

j 

I 


i'OWER  SETTING  AIRSPEED 

ACCELERATION 

SMASH 

EXTENSION 


RELATIVE  ENERGY 
(MDC) 


SMASH 

TIrtical  MANEUVERING 

LAG  PURSUIT 

LO  YO  YO 

LAG  ROLL 

RADIAL  0 

BARREL  ROLL 

CUTOFF 

PURE  PURSUIT 


I 
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(Table  28  continued) 


ANGLE  OFF  SUBASSEMBLY 

ANGLE  OFF  QUARTER  PLANE 

SNAPSHOT 

weapons" parameters 

OVERTAKE 
CUTOFF 
3-9  LINE 
LAO  ROLL 
BARREL  ROLL 


(Table  26  continued) 


LAO  ROLL  (MDC) 


LAO  PURSUIT 


ASPECT  ANGLE 


LAG  ROLL  CONNECTED  COMPONENT 


LAG  PURSUIT 
ASPECT  ANGLE 
G  LOADING 
RELATIVE  ENERGY 
BARREL  ROLL 
3-9  LINE 

WEAPONS  PARAMETERS 
ANGLE  OFF 

VERTICAL  MANEUVERING 
AIRSPEED  ASSEMBLY 

LAG  ROLL 
QUARTER  PLANE 
RELATIVE  ENERGY 
RADIAL  G 
HEAT 

BARREL  ROLL 
6  O'CLOCK 
HEAT  ASSEMBLY 

LAO  ROLL 
3-9  LINE 
HI  Y51S 
QUARTER  PLANE 
CUTOFF 

WEAPONS  PARAMETERS 
BARREL  ROLL 
LO  YO  YO 
SNAPSHOT 
GUNS  ASSEMBLY 


UPTs  IPe  OP 3 

X  X 

X 
X 

X 
X 
X 

X 
X 


X  X 

X 

X 

X 


X 

X 

X 

X 

X 

X 

X  X 

X 


G  LOADING  LAO  ROLL  X 

RADIAL  0  XX 

CORNER  VELOCITY  X 

LIFT  VECTOR  X 

ACCELERATION  X 

WEAPONS  PARAMETERS  X 

SNAPSHOT 
BARREL  ROLL 

EXTENSION  X 

SMASH  X 

AIRSPEED  ASSEMBLY  X  X 

GUNS  ASSEMBLY  "  X 
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|x|x  X  X  x  xlxix  lx  X  X  X  x  xlxlx  lx  X  x  |x  lx  x  xlxixlx 


RADIAL  G  (MDC) 


3-9  LINE  (MDC) 


(Table  28  continued) 


G  LOADING 

WEAPONS  PARAMETERS 
LAG  PURSUIT 
BARREL  ROLL 
LIFT  VECTOR 
QUARTER  plane 
RELATIVE  ENERGY 
VERTICAL  ASSEMBLY 

AHALi  WTBsBHbu 


x 

x 

x 

X 


X 

X 


ASPECT  AN OLE 
HI  YO  YO 
QUARTER  PLANE 
LAG  ROLL 
CUTOFF 
SNAPSHOT 

VERTICAL  MANEUVERING 
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(Table  28  oontinued) 


HEAT 


6  O'CLOCK 


AJCELERATION 


EXTENSION 


HEAT  CONNECTED  COMPONENT 
- uPTa 

6  O'CLOCK 

WEAPONS  PARAMETERS  X 

SMASH  * 

LAG  PURSUIT 
PURE  PURSUIT 
SWITCHOLOGY 


IPa 

X 

X 

X 

X 

X 


TOlf  PURSUIT  x 

OUNS  x 

WEAPONS  PARAMETERS 
LAG  PURSUIT 
QUARTER  PLANE 

VERTICAL/AIRSPEED  ASSEMBj.  X 


X 


X 

X 


OP  a 
X 


X 

X 

X 

X 

X 


* 


ACCELERATION  CONNECTED  COMPONENT 


EXTENSION 

SMASH 

POWER  SETTING 
LO  TO  YO 
AIRSPEED 
0  LOADINO 
OVERTAKE 
PURE  PURSUIT 
AIRSPEED  ASSEMBLY 


X  x 
X  "  x 
X 


X 


ACCELERATION 

QUARTER  PLANE  X 

AIRSPEED  X 

WEAPONS  PARAMETERS  X 

POWER  SETTINO 
SMASH 
0  LOADING 

AIRSPEED  ASSEMBLY  X 
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(Table  28  oonoluded) 


BARREL  ROLL 


INDEPENDENT  CONCEPTS 


LAG  ROLL 
RADIAL  G 
SNAPSHOT 

VERTICAL  MANEUVERING 
ASPECT  ANGLE 
ANGLE  OFF 
G  LOADING 
RELATIVE  ENERGY 
LAO  PURSUIT 
HI  YO  YO 

VERTICAL  ASSEMBLY 
LAG  ASSEMBLY 


UPTs  IPs  QPa 

X 
X 
X 

X  X 

X 

X 

X 

X 

X 

X 

XXX 

XXX 


4 


i 


1 


OVERTAKE 


AIRSPEED 
ANGLE  OFF 
CUTOFF 

LEAD  PURSUIT 

SMASH 

SNAPSHOT 

ACCELERATION 

GUNS  ASSEMBLY  (CUTOFF) 

nrapfefet)  AgsgMBLY 


X 

X 

X 

X 


X 


l 


X 

X 

X 


X 


X 

X 


i 

i 

j 

•: 

,4 

V 


LO  YO  YO 


HI  YO  YO  X 

QUARTER  PLANE  X 

RELATIVE  ENERGY  X 

ACCELERATION 

AIRSPEED 

ASPECT  ANOLE 

CUTOFF 

AIRSPEED/ACCEL.  MACRO. 


X 

X 


X 

X  X 


J 

1 

i 


1 

1 


Table  28  also  reveals  connections  from  concepts  to  assemblies.  To 
connect  from  a  concept  to  an  assembly  requires  only  a  link  from  that 
concept  to  any  concept  in  that  assembly.  For  example,  WEAPONS  PARAM¬ 
ETERS  has  a  critical  link  to  the  Heat  Assembly.  This  means  that  while 
IPs  and  GPs  did  not  agree  on  the  direct  link  emanating  from  WEAPONS 
PARAMETERS,  they  did  agree  that  there  should  be  some  connection  from 
WEAPONS  PARAMETERS  to  the  assembly  of  concepts  dealing  with  Heat.  In 
this  specific  example,  the  MEN  for  IPs  had  a  WEAPONS  PARAMETERS  to 
HEAT  link  while  the  GPs  had  a  link  between  WEAPONS  PARAMETERS  and  6 
O’CLOCK.  Since  HEAT  and  6  O’CLOCK  belong  to  the  same  assembly,  a  link 
is  added  to  that  assembly.  This  is  incidentally  a  critical  link  that 
the  UPTs  also  have. 

Table  28  permits  the  classification  of  concepts  along  two  dimen¬ 
sions:  underdefinition  and  overdefinition.  Underdefined  concepts  for 
UPTs  are  those  that  are  missing  a  large  proportion  of  critical  links. 
Overdefined  concepts  are  those  that  the  UPTs  have  connected  to  the 
network  using  links  that  neither  the  IPs  nor  the  GPs  have  in  their 
networks.  We  determined  the  probability  of  a  UPT  having  a  link 
£l  yen  that  the  link  was  critical  as  well  as  the  probability  of  the 
UPTs  having  a  link  given  that  neither  experienced  pilot  group  had  the 
link.  We  then  did  a  median  split  on  each  dimension  and  classified  the 
concepts  as  to  whether  they  were  high  on  both  dimensions,  low  on  both 
dimensions,  or  high  on  one  and  low  on  the  other.  This  leads  to  four 
classes  of  concepts:  well  defined,  over  defined,  under  defined,  and 
misdeflned.  These  concepts  appear  in  Table  29.  The  actual  links 
involved  in  the  classification  of  these  concepts  appear  in  Table  30. 

The  critical  links  not  held  by  tne  UPTs  are  those  relations  found 
in  the  conceptual  structure  of  experienced  pilots  but  not  in  that  of 
the  students.  Some  of  the  structure  missing  in  the  student  networks 
are  concept-to-concept  links  (e.g.,  VERTICAL  MANEUVERING-- QUARTER 
PLANE)  whereas  others  are  concept- to-assembly  links  (e.g.,  LAG 
PURSUIT — HEAT  ASSEMBLY).  Recall  that  if  a  student  is  missing  a 
critical  concept- to-assembly  link,  the  MEN  for  the  UPTs  does  not 
contain  a  link  from  that  concept  to  any  concept  in  that  assembly.  For 
some  concepts,  the  UPTs  have  none  of  the  critical  relations:  LEAD 
PURSUIT.  HEAT,  and  LOW  Y0  Y0.  In  fact,  for  LEAD  PURSUIT,  the  UPTs 
show  no  link  to  any  concept  in  the  Guns  Subassembly  to  which  it 
belongs.  Two  other  critical  relations  shown  in  Table  30  are  worth 
highlighting  here.  Both  the  VERTICAL  MANEUVERING— QUARTER  PLANE  and 
the  ACCELERATION— EXTENSION  relations  are  highly  discriminating 
relations  according  to  the  pattern  classification  analysis  discussed 
earlier.  The  fact  that  these  two  relations  are  critical  (according  to 
the  OWN  analysis)  and  the  fact  that  they  are  important  in  discrim¬ 
inating  between  novice  and  experienced  pilots  (according  to  the 
pattern  classification  analysis)  provide  convergent  validation  that 
these  relations  are  important  parts  of  an  expert’s  organization  of 
flLght  information  that  In  lacking  in  that  of  the  novice. 
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Table  29 


Comparison  of  Novioe  and  Expert 


Well-Defined  Conoegta 


GUNS 

0  LOADING 


PURE  PURSUIT 
LIFT  VECTOR 


AIRSPEED 


OVERTAKE 


Over-Defined  Conoeota  ( Compare  Table  30) 


SNAPSHOT  WEAPONS 

VERTICAL  MANEUVERING  HI  YO  TO 
BARREL  ROLL  SMASH 

LAG  PURSUIT  3-9  LINE 

EXTENSION  LO  YO  YO 


Under-Defined  Conoeota  (Compare  Table  30) 

LEAD  PURSUIT  CUTOFF 

ASPECT  ANOLE  HEAT 


Miadefined  Conoeota 

LAG  ROLL  RADIAL  0 
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Concepts 


SWITCHOLOGY 
6  0* CLOCK 
CORNER  VELOCITY 


ANGLE  OFF 
QUARTER  PLANE 
RELATIVE  ENERGY 
ACCELERATION 


POWER  SETTING 


thiEmbkiiktauaA  ait 


JJ-‘-—|iiia'lrnH  I  nil  u  ni 


Table  30 


Differences  In  Conoept  Links  for  Novices  and  Experts 


Critioal  Expert  Links  not  _in  Novice  Network 
(Underdefined  Concepts) 


QUM3-LEAD  PURSUIT 
CUTOFF-LEAD  PURSUIT 
VERTICAL  MANEUVERINO-AIRSPEED 
LAQ  ROLL-ASPECT  ANGLE 
LAO  ROLL-AIRSPEED  ASSEMBLY 
ASPECT  ANOLE-3/9  LINE 
RADIAL  Q-VERTICAL  ASSEMBLY 
HEAT-6  O'CLOCK 

LO  YO  YO-AIRSPEED/ACCELERATION 
MACROASSEMBLY 


SNAPSIIOT-ANOLE  OFF 

VERTICAL  MANKUVERING-QUARTER  PLANE 

AIRSPEED-POWER  SETTING 

LAG  ROLL-C  LOADING 

LAG  PURSUIT-HEAT  ASSEMBLY 

0  LOADING-GO!1 3  ASSEMBLY 

RADIAL  G-ANGLE  OFF  ASSEMBLY 

ACCELERATION-EXTENSION 


Links  in  the  Novloe  Network  not  found  for  Ex  arts 
(Overdefined  Concept aV  ~ 


OUNS-6  O'CuOCK 
BARREL  ROLL-SNAPSHOT 
WEAPONS  PARAMETERS-LAQ  ROLL 
WEAPONS  PARAMETBRS-Q  LOAD1NO 
ANGLE  OFF-OVERT AKE 
HI  YO  YO-LO  YO  YO 
LIFT  VECTOR-QUARTER  PLANE 
QUARTER  PLANE-ASPECT  ANGLE 
SMASH-HEAT 

RELATIVE  ENEROY-LAO  PURSUIT 
LAQ  ROLL-BARREL  ROLL 
LAQ  PURSUIT-RADIAL  0 


3/9  LINE-SNAPSHOT 
WEAPONS  PARAMETERS-EXTEN3I0N 
WEAPONS  PARAMETERS-RADIAL  0 
CUTOFF-3/9  LINE 

VERTICAL  MANEUVERINO-RELATIVE  ENERGY 
HI  YO  YO-  3/9  LINE 
QUARTER  PLANE-EXTENSION 
QUARTER  PLANE-LO  YO  YO 
POWER  SETTING-ACCELERATION 
RELATIVE  ENEROY-LO  YO  YO 
LAO  ROLL-3/9  LINE 
RADIAL  Q-BARREL  ROLL 
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Perhaps  as  serious  as  not  having  the  oorreot  relations  is  having 
inappropriate  relations.  The  overdefined  oonoepts  are  those  that  the 
students  have  oonneoted  to  the  network  In  a  fashion  that  is  different 
from  either  group  of  experienced  pilots.  For  some  oonoepts,  this 
overdefinition  is  quite  severe:  WEAPONS  PARAMETERS,  QUARTER  PLANE,  and 
3-9  LINE.  In  faot,  for  QUARTER  PLANE,  19)1  (N/21)  of  those  relations 
not  found  in  an  experienced  pilot’s  network  are  inoluded  in  the  UPT 
structure.  One  other  inappropriate  relation  is  worth  highlighting. 
The  pattern  classification  procedures  suggest  that  this  relationship 
is  highly  discriminating,  and  the  network  analysis  suggests  that  it  is 
important  to  the  *JPT  structure:  HIOH  YO  YO— LOW  YO  YO.  Apparently  the 
similarity  between  HIOH  and  LOW  YO  YO  la  in  name  only  for  the  exper¬ 
ienced  pilots,  yet  the  UPTs  have  that  as  a  direct  ooncept-to-oonoept 
link  in  their  network. 

Finally,  two  oonoepts  did  not  inolude  many  of  the  oritloal  links 
and,  in  addition,  inoluded  many  extraneous  links.  In  this  sense,  LAG 
ROLL  and  RADIAL  Q  were  both  underdefined  and  overdefined,  but  more 
appropriately  we  have  termed  them  miadefined.  For  LAO  ROLL,  the  UPTs 
do  not  have  oritioal  links  to  ASPECT  ANGLE ,  0  LOADINO,  nor  any  link  to 
the  AIRSPEED  SUBASSEMBLY.  Instead  they  do  have  extraneous  links  from 
LAO  ROLL  to  BARREL  ROLL,  3-9  LINE,  and  WEAPONS  PARAMETERS.  Similarly, 
for  RADIAL  G  the  UPTs  have  no  oritioal  oonneotiona  to  any  oonoept  in 
either  the  VERTICAL  SUBASSEMBLY  or  the  ANGLE  OFF  ASSEMBLY,  but  do  have 
extraneous  links  to  WEAPONS  PARAMETERS,  LAG  PURSUIT,  and  BARREL  ROLL. 

This  phase  of  the  projeot  has  highlighted  those  oonoepts  and 
relations  that  the  UPTs  have  not  as  yet  mastered.  The  emphasis  on 
differenoea  between  the  UPTs  and  expert  pilots  should  not  be  taken  as 
indicative  of  the  progress  of  the  UPTs,  In  faot,  their  overall 
oorrespondenoe  with  the  IPs,  and  especially  with  the  GPs,  is  quite 
high.  Several  oonoepts  are  quite  well  mastered  while  several  others 
are  only  slightly  (1  link)  different  from  the  more  experienced  pilots. 

The  use  of  general  networks  and  the  OWN  algor l tm  holds  substan¬ 
tial  promise  in  attempts  to  specify  the  local  relations  and  structure 
present  in  the  oonoeptual  organization  of  oritioal  flight  inform¬ 
ation.  In  addition,  it  allows  a  detailed,  oonoept-by-oonoept,  compar¬ 
ison  aoross  groups  that  differ  in  expertise.  The  use  of  networks 
ooupled  with  other  techniques,  especially  MDS  and  pattern  classif¬ 
ication,  should  provide  valid,  usable  information  in  a  form  that  oould 
be  incorporated  into  the  training  prooedureo  of  fighter  pilots. 


GENERAL  DISCUSSION 


Primary  Accomplishments 

This  project  has  produced  several  interesting  and  potentially 
useful  findings*  The  major  new  theoretical  contribution  lies  in  the 
development  of  methods  for  obtaining  and  analyzing  networks  of 
concepts.  This  method  should  contribute  to  the  continued  development 
of  an  understanding  of  structures  and  processes  in  semantic  memory. 

In  the  domain  of  critical  flight-related  concepts,  we  have  3hown 
that  systematic  methods  can  yield  valid  and  reliable  descriptions  of 
the  structure  of  these  concepts.  Further,  these  structures  can  be 
used  to  identify  individuals  as  members  of  groups  with  different 
training  and  experience.  We  have  also  identified  some  specific  areas 
of  disagreement  in  the  structures  possessed  by  expert  and  novice 
pilots.  These  specific  differences  may  deserve  special  attention  in 
lead-in  fighter  training.  The  structures  themselves  may  also  prove  to 
be  useful  in  the  academic  program  since  they  provide  some  graphic 
examples  of  the  differences  in  the  ways  novices  and  experts  think 
about  critical  flight  concepts. 

In  the  context  of  the  ongoing  effort  in  this  contract,  this  first 
contract  period  has  resulted  in  several  analytical  methods  with  which 
to  pursue  experimental  analysis  of  conceptual  structures  in  fighter 
pilots.  While  much  of  the  initial  effort  has  been  concerned  with 
measurement  issues,  it  has  been  necessary  to  define  and  validate  the 
basic  purpose  of  the  contract.  The  work  in  the  initial  period  has 
also  led  to  several  structural  analyses  that  serve  to  define  struct¬ 
ures  of  memory  for  critical  flight  information.  We  now  have  a  firm 
foundation  on  which  to  build  further  work.  While  we  intend  to 
continue  to  investigate  the  problems  associated  with  representing 
knowledge,  the  next  contract  period  will  include  experimental 
investigations  of  the  structures  we  have  already  established. 

Utility  and  Limitations  of  Particular  Structural  Analyses 

In  this  project,  several  techniques  have  been  applied  for 
obtaining  structural  representations  of  critical  flight  information. 
We  began  by  applying  traditional  scaling  techniques,  such  as  multi¬ 
dimensional  scaling  and  hierarchical  clustering  analysis  to  psycho¬ 
logical  distances  derived  from  similarity  ratings.  We  then  developed 
a  new  structuring  technique,  general  weighted  networks,  and  applied  it 
to  the  same  similarity  data.  Though  the  resulting  critical  flight 
structures  point  to  a  number  of  similar  conclusions,  they  do  differ  in 
a  number  of  ways.  Consideration  of  these  differences  in  light  of  the 
current  interest  in  fighter  pilot  conceptual  memory  will  allow  the 
important  details  to  be  abstracted  from  each  structure.  These  details 
can  be  used  to  gain  an  understanding  of  critical  flight  information 
that  could  not  be  gleaned  from  any  3 ingle  structural  representation. 
We  believe  that  the  appropriate  U3c  of  nil  structural  representations 


-8U- 


is  the  most  sensible  way  to  proceed  in  attempting  to  understand  any 
organized  oonoeptual  domain.  It  is  a  particularly  rational  approach 
in  attempting  to  understand  oonoeptual  domains  that  have  been 
subjected  to  almost  no  prior  analysis,  as  is  the  oase  with  orltioal 
flight  information. 

Consider  first  the  results  of  the  Multldimenslnal  Soaling  (MDS ) 
procedures.  MDS  took  as  input  a  set  of  empirical  psychological 
distances  and  produced  a  set  of  derived  distances.  The  derived 
dlstanoes  contained  metrio  properties  that  ware  not  present  in  the 
emplrioal  Judgments.  The  faot  that  this  metrio  information  is  quite 
useful  is  highlighted  by  the  consistent  improvement  in  correlations 
found  with  dlstanoes  oompared  to  emplrioal  ratings  and  in  the  more 
suooeasful  pattern  classification  found  with  the  MDS  distances.  The 
constraints  that  MDS  plaoes  on  the  distances  are  quite  severe  (e.g., 
the  formation  of  iaoaoelea  triangles)  and  thus  results  in  both  very 
useful  and  very  misleading  information  in  the  struotures.  The  useful 
information  appears  in  the  form  of  global  properties  of  the 
structure.  For  example,  only  MDS  can  abstract  dimensions  from  the 
rating  data,  but  it  oan  produoe  looal  distortions  in  the  dlstanoes 
between  particular  pairs  of  oonoepts. 

With  MDS,  solutions  of  moderate  dimensionality  were  obtained  for 
eaoh  of  the  aubjeot  groups.  Provided  that  the  dimensions  produosd  in 
the  solution  oan  be  identified,  these  dimensions  oan  provide  a  great 
deal  of  information  about  the  underlying  oharaoteristia  features  of 
the  oonoeptual  domain.  For  example,  the  faot  that  a  temporal  sequenc¬ 
ing  seems  to  pervade  these  domains  is  obviously  orltioal  to  the 
understanding  of  the  structure.  The  temporal  dimension  serves  as  a 
good  example  of  both  a  strength  and  weakness  of  multidimensional 
soaling  solutions.  No  other  soaling  solution  oould  have  shown  this 
continuous  feature  of  the  atruoture.  Thus,  MDS  la  unsurpassed  in  its 
ability  to  point  out  Important  underlying  continuous  dimensions  of 
variation  in  the  oonoepts.  Alternatively,  if  the  underlying  dimension 
is  discrete  then  the  MDS  solution  is  less  then  optimal  and  should  be 
used  in  oonjunction  with  one  of  the  other  soaling  procedures.  So, 
though  the  metrio  properties  of  an  MDS  solution  allows  for  the  abstrac¬ 
tion  of  dimensions,  a  boundary  oondition  on  the  usefulness  of  this 
global  feature  of  MDS  is  that  the  dimension  be  oontlnuoua. 

Though  the  MDS  metrio  plaoes  points  in  an  S'  dimensional  apaoe  and 
allows  for  the  identification  of  continuous  dimensions,  the  metric 
requires  a  oertain  saorifioe  to  aooomplish  this,  The  saorifice  is  in 
terms  of  local  distortions  in  the  struoture  whloh  prevent  an  analysis 
of  the  subcomponents  of  the  MDS  struoture.  Hierarchical  Clustering 
(HC)  analysis  and  General  Weighted  Networks  (OWN)  foous  on  these  looal 
relationships,  though  they  provide  less  global  Information  than  does 
MDS. 
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HC  tranforms  the  empirical  ratings  into  a  set  of  Glasses.  A 
conoept  within  a  cluster  is  more  related  to  another  member  of  the 
cluster  than  to  a  conoept  from  another  cluster.  Comparing  an  HC 
solution  with  an  MDS  solution  for  the  same  data  helps  point  out  some 
of  the  distortions  that  MDS  makes  in  the  local  structure.  Two  items 
close  together  in  multidimensional  space  may  not  cluster  together 
while  two  items  distant  in  space  may  cluster  together.  Figure  17 
shows  an  HC  solution  superimposed  on  a  two  dimensional  space  of  split- 
plane  concepts  for  IPs. 

HC  requires  that  a  oonoept  belong  to  one  and  only  one  cluster 
though  that  duster  as  a  whole  may  belong  to  a  larger,  in  a  sense 
superordinate,  cluster.  This  hierarchical  constraint  also  results  in 
some  distortions  in  the  structure  since  many  oonoepts  do  not  exist 
with  one  another  in  a  hierarchical  relationship.  Another  problem  with 
using  HC  as  a  means  for  determining  local  relations  is  that  there  is 
no  a  priori  way  to  partition  the  tree  to  allow  finer  grained 
analyses.  Though  the  trees  for  UPTs  and  IPs  were  partitioned  into 
five  major  dusters,  this  oouuld  have  easily  been  four  or  seven. 
General  weighted  networks  provide  a  better  vehiole  for  determining 
local  relations.  However,  HC  does  produce  free  nodes  in  its  solution 
(unlabelled  nodes  in  the  tree)  that  GWN  does  not  produoe  in  its 
present  form.  These  free  nodes  may  be  valuable  if  one  wished  to 
determine  hierarchical  relations,  though  they  are  often  diffioult  to 
label.  The  OWN  does  not  distinguish  between  hierarohioal  and 
nonhierarohioal  relations  without  some  a  priori  knowledge  of  the 
relationship  between  oonoepts.  However,  MDS,  HC,  and  GWN  are  not 
typioally  applied  to  random  collections  of  oonoepts;  rather  the  user 
typically  brings  intuition  to  bear  on  the  establishment  of  the 
oonoeptual  domain.  Thus  there  is  no  apparent  benefit  or  HC  over  GWN 
for  establiahing  local  relations  within  a  conceptual  structure. 

GWN  also  takes  psychological  distance  data  as  input  and  produoes 
a  oonoeptual  structure  in  the  form  of  a  network.  Currently,  the 
network  mirrors  the  original  ratings  rather  thBn  transforming  them  in 
any  way.  This  has  the  advantage  of  staying  close  to  the  data  base 
and,  thus,  not  Introducing  distortion  into  the  links.  However,  there 
is  distortion  in  ohains  of  links  in  that  the  original  ratings  do  not 
correspond  to  the  length  of  a  chain.  Future  work  will  focus  on 
Improving  the  metrio  properties  of  GWN. 
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Barrel  flolll 


Concepts  enclosed  in  loops  duster  together.  Some  nearby  concepts 
dc  not  Cluster  and  some  distant  concepts  do  duster . 


Networks  have  a  number  of  properties  that  allow  for  a  better 
understanding  of  conceptual  structures.  Unlike  MDS,  networks  focus  on 
the  looal  relations  of  the  structure.  In  this  sense,  it  is  similar  to 
HC.  Networks  allow  multiple  connections  within  the  structure  and  thus 
allow  one  to  partition  the  network  into  assemblies  and  cycles.  As  has 
been  shown  in  this  report,  assemblies  and  cycles  form  meaningful  units 
that  aid  in  the  understanding  of  the  overall  network.  In  addition, 
cycles  and  assemblies  allow  a  comparison  across  groups  of  individuals 
on  a  concept-by-concept  basis:  a  task  that  is  difficult  or  impossible 
to  do  with  the  more  traditional  scaling  procedures.  Finally,  networks 
are  weighted  and  this  distance  information  can  be  used  to  form  minimum 
cycles,  determine  the  concept  in  the  network  that  is  closest  to  all 
other  concepts  (mediae),  or  determine  the  concept  that  allows  the  most 
rapid  connection  to  the  most  distant  concept  (center).  Though  these 
last  two  properties  are  more  interpretable  in  a  metricized  network, 
their  potential  for  summarizing  networks  and  differences  among 
networks  can  be  quite  useful.  Finally,  networks  potentially  allow 
concepts  as  well  as  links  to  be  weighted.  More  important  concepts  or 
more  ” costly”  concepts  could  be  weighted  in  such  a  way  as  to  make  some 
concepts  more  critical  than  others.  As  GWN  now  stands,  it  provides  a 
great  deal  of  information  about  the  local  structure  of  conceptual 
domains  and,  with  the  addition  of  a  metric,  will  introduce  a  number  of 
other  features  that  will  aid  in  understanding  critical  flight  inform¬ 
ation.  We  believe  that  the  results  of  GWN,  coupled  with  the  global 
characteristics  gleaned  from  MDS  analyses,  should  provide  the  best 
understanding  of  the  structure  of  critical  flight  information. 

Directions  for  Future  Work 

One  major  area  to  be  pursued  in  future  work  concerns  the 
experimental  verification  of  the  structures  we  have  identified.  We 
intend  to  pursue  this  goal  with  priming  methodology  and  recognition 
memory  experiments.  We  are  also  proceeding  with  the  analysis  of 
memory  in  the  form  of  scripts  and  frames.  It  would  also  be  useful  to 
pursue  the  predictive  power  of  these  techniques  by  systematically 
following  individual  UPTs  through  training.  Perhaps  individuals  could 
be  identified  who  would  most  benefit  from  fighter  braining. 

The  network  algorithm  should  be  developed  further.  It  should  be 
possible  to  develop  a  metric  based  on  the  network' that  could  lead  to 
some  experimental  comparisons  of  the  MDS  metric  and  the  GWN  metric. 
While  the  network  itself  is  produced  by  an  algorithm,  more  work  is 
needed  to  develop  algorithms  for  defining  cycles  and  assemblies  in  the 
networks.  In  short,  the  network  representation  is  most  promising,  and 
additional  efforts  will  be  required  for  the  network  analysis  to  reach 
its  full  potential. 
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Further  detailed  analysis  of  the  eonoepts  from  the  low-angle 
strafe  maneuver  could  be  performed  along  the  same  lines  as  that 
presented  here  for  the  split-plane  maneuvers.  Also,  another  group  of 
students  who  have  completed  fighter  lead-in  training  should  be 
obtained  to  permit  an  analysis  of  the  effects  of  training  in  the 
speolflc  maneuvers  represented  by  our  stimulus  materials. 

Recommendations 

The  researoh  presented  in  this  report  provides  a  detailed 
analysis  of  the  conooptual  structure  of  oritioal  flight  information 
in  fighter  pilots.  As  such,  the  structures  should  be  of  use  in 
designing  training  programs  for  students  attempting  to  acquire  these 
oonoeptual  struoturea.  In  addition  the  representations  themselves 
may  prove  to  be  useful  as  training  aids.  The  network  analysis,  for 
example,  shows  how  expert  fighter  pilots  organize  the  oonoepts 
involved  in  particular  maneuvers.  To  the  extent  that  the  network 
representation  provides  an  understandable  representation  of  experts' 
knowledge  structures,  students  may  find  it  useful  in  learning  about 
the  maneuvers. 

From  a  somewhat  different  angle,  we  have  identified  specific 
differences  in  the  oonceptuel  struoturea  of  students  and  expert 
fighter  pilots.  In  partloular,  the  differences  show,  in  part,  what 
experts  know  that  students  do  not  and  what  misconceptions  the  students 
may  have  acquired  rrom  earlier  training  or  from  other  life  exper¬ 
iences.  These  areas  of  difference  should  receive  speoiel  attention  in 
the  training  program  for  fighter  pilots. 

Finally,  our  work  in  classifying  individuals  based  on  their 
oonoeptual  structure  suggests  further  work  in  attempting  to  predict 
the  suooess  of  future  fighter  pilots  based  on  the  oonoeptual 
struoturea  students  demonstrate  early  on  in  training.  It  may  be 
neoessary  to  study  the  structures  associated  with  a  different  set  or 
oonoepts  than  those  used  in  the  present  investigation.  For  example, 
perhaps  some  concepts  relating  to  attitude  and  motivation  should  be 
included  along  with  oonoepts  relating  to  the  operation  of  aircraft. 
The  olaaslf loation  techniques  we  have  developed  appear  to  be  very 
sensitive  to  differences  in  cognitive  structures,  and  they  may  well 
provide  some  prediotive  power  for  organizing  the  training  program  to 
produce  maximum  benefit  for  those  who  are  likely  to  benefit  the  moot 
from  fighter  training. 
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APPENDIX 


The  tables  in  the  appendix  present  the  detailed  inter-individual 
correlations  for  both  rating  soores  and  MDS  distanoes.  Both  the  aplit- 
blane  concepts  and  the  low-angle  strafe  concepts  are  represented. 


Split-Plane  Manuevera 
Inter-Individual  Correlation  Matrix 
II  through  17  are  individual  instructs  pilots 


Rating  Soores 


11 

12 

13 

14 

15 

16 

17 

11 

.35 

.47 

.45 

.42 

.58 

.43 

12 

.35 

- 

.41 

.41 

.47 

.38 

.32 

13 

.47 

.41 

.49 

.43 

.42 

.43 

in 

.**5 

.41 

.49 

- 

.41 

.43 

.41 

15 

.42 

.47 

•  43 

.41 

MB 

.42 

.41 

16 

.58 

.38 

.42 

.43 

.42 

- 

.37 

17 

.43 

.32 

.43 

.41 

.41 

.37 

- 

Mean  Correlation  ■  .M2 


Distanoes  in  Multidimensional  Spaoe 


11 

12 

13 

14 

15 

16 

17 

11 

.34 

.53 

.50 

.45 

.57 

.47 

12 

.34 

- 

.41 

.40 

.43 

.35 

.38 

13 

.53 

.41 

- 

.59 

.47 

.44 

.52 

14 

.50 

.40 

.59 

- 

.41 

.51 

.51 

15 

.45 

.43 

.47 

.41 

- 

.41 

.49 

16 

.57 

.35 

.44 

.51 

.41 

M 

.37 

17 

.47 

.38 

.52 

.51 

.49 

.37 

- 

Mean  Correlation  ■  .MS 
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Split-Plane  Manuevers 
Inter-Individual  Correlation  Matrix 
01  through  G9  are  individual  guard  pilots 


Rating  Scores 


G1 

02 

G3 

G4 

G5 

G6 

G7 

G8 

09 

01 

.44 

.42 

.32 

•  30 

.37 

.34 

.38 

.36 

02 

.44 

- 

.40 

.36 

.42 

.43 

.51 

.43 

.36 

03 

.42 

.40 

- 

•  33 

.29 

.40 

.43 

.31 

.39 

04 

.32 

.36 

.33 

- 

.32 

.38 

.25 

.42 

.33 

05 

.30 

.42 

.29 

.32 

- 

.38 

.31 

.29 

•  33 

06 

.37 

.43 

.40 

.38 

.38 

- 

.30 

.46 

.36 

07 

.34 

.51 

.43 

.25 

.31 

.30 

- 

.33 

.38 

08 

.38 

.43 

.31 

.42 

.29 

.46 

.33 

- 

.25 

09 

.36 

.36 

.39 

.33 

.33 

.36 

.38 

.25 

- 

Mean  < 

Correlation 

1  ■  . 

36 

Diatanoea  in 

Multidimensional  Spaoe 

01 

02 

03 

G4 

05 

06 

07 

08 

09 

01 

m 

.40 

.47 

•  37 

.32 

.34 

.39 

.38 

•  37 

02 

.40 

.47 

.42 

.41 

.41 

.50 

.40 

.38 

03 

.47 

.47 

m 

.42 

.32 

.43 

.54 

.33 

.50 

04 

.37 

.42 

.42 

m 

.35 

.43 

.35 

.41 

.38 

05 

.32 

.41 

.32 

.35 

. 

.32 

.38 

.27 

.33 

06 

•  34 

.41 

.43 

.43 

.32 

- 

.34 

.45 

.36 

07 

.39 

.50 

.54 

.35 

.38 

.34 

- 

.32 

.47 

08 

.38 

.40 

.33 

.41 

.27 

.45 

.32 

- 

.29 

09 

.37 

.38 

.50 

.30 

.33 

.36 

.47 

.29 

- 

Mean 

Correlation  » 

.39 
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Split-plane  Manuevera 
Inter-Individual  Correlation  Matrix 
U1  through  U 1 7  are  individual  undergraduate  pilota 

01  02  03  04  U5  U6  U7  U8  U9  010  Oil  U12  U13  014  U15  U 16  U 1 7 


Hating  Spores 


U1 

- 

.31 

.24 

.29 

.37 

.30 

.14 

.28 

.44 

.34 

.36 

.40 

.31 

.44 

.35 

.29 

.24 

U2 

.31 

- 

.26 

.32 

.26 

.25 

.12 

.21 

.34 

.30 

.34 

.29 

.26 

.36 

.28 

.28 

.28 

03 

.24 

.26 

- 

.32 

.37 

.31 

.26 

.44 

.37 

.29 

.44 

.36 

•  35 

.38 

.27 

.30 

.23 

04 

.29 

.32 

.32 

- 

.39 

.30 

.12 

.30 

.45 

.33 

.47 

.29 

.30 

.49 

.27 

.35 

.28 

05 

.37 

.26 

.37 

.39 

- 

.20 

.22 

.28 

.42 

.24 

.38 

.41 

.26 

.52 

.27 

.29 

.30 

U6 

.30 

.25 

.31 

.30 

.20 

- 

.25 

.34 

.33 

.24 

.30 

.26 

.29 

.35 

.25 

.18 

.28 

07 

.14 

.12 

.26 

.12 

.22 

.25 

- 

.24 

.28 

.30 

.30 

.34 

.30 

.32 

.17 

.23 

.28 

U8 

.28 

.21 

.44 

.30 

.28 

.34 

.24 

m 

.36 

.26 

.43 

.24 

.41 

.36 

.22 

.19 

.21 

U9 

.44 

.34 

.37 

.45 

.42 

.33 

.28 

.36 

. 

.37 

.43 

.34 

.36 

.46 

.31 

.30 

.29 

U10 

.34 

.30 

.29 

.33 

.24 

.24 

.30 

.26 

.37 

- 

.34 

.36 

.31 

.36 

.30 

.27 

.26 

U11 

.36 

.34 

.44 

.47 

.38 

.30 

.30 

.43 

.43 

.34 

- 

.36 

.43 

.43 

.38 

.32 

.29 

U12 

.40 

.29 

.36 

.29 

.41 

.26 

.34 

.24 

.34 

.36 

.36 

- 

.32 

.47 

.31 

.29 

.27 

U 1 3 

.31 

.26 

.35 

.30 

.26 

.29 

.30 

.41 

.36 

.31 

.43 

.32 

.39 

.29 

.29 

.29 

U1M 

.44 

.36 

.38 

.49 

.52 

.35 

.32 

.36 

.46 

.36 

.43 

.47 

.39 

• 

.33 

.35 

.35 

U 1 5 

.35 

.28 

.27 

.27 

.27 

.25 

.17 

.22 

.31 

.30 

.38 

.31 

.29 

.33 

*» 

.18 

.26 

U 1 6 

.29 

.28 

.30 

.35 

.29 

.18 

.23 

.19 

.30 

.27 

.32 

.29 

.29 

.35 

.18 

m 

.34 

U 17 

.24 

.28 

.23 

.28 

.30 

.28 

.28 

.21 

.29 

.26 

.29 

.27 

.29 

.35 

.26 

.34 

- 

Mean  Correlation  ■  .31 


Dlatanoea  in  Multldimenaional  Spaoe 


U1 

- 

.33 

.28 

.28 

.33 

.27 

,24 

.31 

.46 

.39 

.39 

.30 

.29 

.43 

.30 

.32 

.26 

U2 

.33 

- 

.27 

.27 

.18 

.22 

.18 

.19 

.34 

.28 

.29 

.24 

.29 

.36 

.26 

.31 

.26 

U3 

,28 

.27 

m 

•  33 

.35 

.27 

.25 

•  38 

•  38 

.32 

.41 

.28 

.33 

.31 

.26 

.33 

.21 

U4 

.28 

.27 

.33 

- 

.27 

.16 

.12 

.27 

.36 

.34 

.39 

.24 

.33 

.41 

.26 

.29 

.29 

U5 

.33 

.18 

.35 

.27 

- 

.17 

.10 

.31 

.27 

.22 

.32 

.26 

.22 

.44 

.25 

.23 

.25 

06 

.27 

.22 

.27 

.16 

.17 

- 

.29 

.29 

.35 

.19 

.23 

.22 

.27 

.36 

.21 

.15 

.25 

U7 

.24 

.18 

.25 

.12 

.10 

.29 

- 

.20 

.29 

.33 

.32 

.23 

.28 

.25 

.13 

.30 

.27 

U8 

.31 

.19 

.38 

.27 

.31 

.29 

.20 

- 

.35 

.29 

.42 

.21 

•  35 

.35 

.29 

.21 

.19 

09 

.46 

.34 

.38 

.36 

.27 

.35 

.29 

.35 

- 

.38 

.42 

.28 

•  34 

.47 

.33 

.28 

.31 

U1 

.39 

.28 

.32 

.34 

.22 

.19 

.33 

.29 

.38 

- 

.41 

.24 

.31 

.32 

.37 

.30 

.28 

U11 

.39 

.29 

.41 

.39 

.32 

.23 

.32 

.42 

.42 

.41 

- 

.21 

.43 

.35 

.38 

.25 

.26 

U12 

.30 

.24 

.28 

.24 

.26 

.22 

.23 

.21 

.28 

.24 

.21 

- 

.15 

.37 

.20 

.24 

.26 

U 1 3 

.29 

.29 

.33 

.33 

.22 

.27 

.28 

.35 

.34 

.31 

.43 

.15 

.40 

.26 

.29 

.26 

U 1 4 

.43 

.36 

.31 

.41 

.44 

.36 

.25 

.35 

.47 

.32 

.35 

.37 

.40 

- 

.28 

.35 

.43 

U15 

.30 

.26 

.26 

.26 

.25 

.21 

.13 

.29 

.33 

.37 

.38 

.20 

.26 

.28 

- 

.14 

.28 

U 1 6 

.32 

.31 

.33 

.29 

.23 

.15 

.30 

.21 

.28 

.30 

.25 

.24 

.29 

.35 

.14 

• 

.28 

017 

.26 

.26 

.21 

.29 

.25 

.25 

.27 

.19 

.31 

.28 

.26 

.26 

.26 

.43 

.28 

.28 

- 

Mean  Correlation  * 
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Split-Plane  Manuevera 


Intar-Individual  Correlation  Matrix 
W1  through  W4  are  individual  IWSOa 


Rating  Scores 


W1 

W2 

W3 

W4 

W1 

m 

.45 

.42 

.40 

W2 

.45 

- 

.24 

.28 

W3 

.42 

m 

.50 

W4 

•  DO 

.28 

.50 

. 

Mean  Correlation  ■  .38 
.atanoea  ijj  Multidimensional  Sj 


W1 

W2 

W3 

W4 

W1 

«• 

.36 

.46 

.45 

W2 

.36 

- 

.24 

.23 

W3 

.46 

.24 

M 

.53 

W4 

.45 

.23 

.53 

- 

Mean  Correlation  *  .38 


Split-Plane  Manuevera 

Inter-Individual  Correlation  between  Guard 
and  Instructor  Pilots 

G1  through  09  are  individual  guard  pilots 
II  through  17  are  individual  inatruotor  pilots 


Rating  Spores 


01 

02 

G3 

G4 

G5 

G6 

07 

08 

G9 

11 

.38 

.42 

.46 

.38 

.32 

.34 

.38 

.42 

.3** 

12 

.36 

.33 

.35 

.21 

.25 

.24 

.31 

.16 

.25 

13 

.40 

.41 

.39 

.30 

.30 

.28 

.42 

.32 

.44 

14 

.38 

.39 

.41 

.31 

.37 

.33 

.42 

.28 

.35 

15 

.38 

.36 

.44 

,27 

.24 

.29 

.39 

.26 

.45 

16 

.34 

.31 

.40 

.29 

.24 

.32 

.30 

.28 

.32 

17 

.39 

.46 

.43 

.37 

.36 

.35 

.48 

.25 

.43 

Mean  Correlation  «  .35 
Dlstanoea  in  Multidimensional  Sptoe 


□  1 

02 

03 

04 

05 

06 

07 

08 

09 

11 

.38 

.40 

.50 

.44 

.36 

.39 

.44 

.36 

.39 

12 

.37 

.31 

.36 

.30 

.29 

.27 

.39 

.23 

.40 

13 

.42 

.45 

.47 

.29 

.35 

.30 

.48 

.30 

.50 

14 

.41 

•  39 

.45 

.33 

.38 

.30 

.45 

.24 

.42 

15 

.32 

.36 

.48 

.27 

.25 

.32 

.41 

.25 

.50 

16 

.37 

•  33 

.41 

.32 

.25 

.31 

.28 

.28 

.32 

17 

.42 

.51 

.52 

.40 

.36 

.35 

.49 

.29 

.46 

Mean  Correlation  «  .37 
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Split-Plane  Manuevers 

Inter-Individual  Correlations  for  IPs  and  UPTa 

II  through  17  are  Individual  instructor  pilots 
U1  through  U17  are  individual  undergraduate  pilots 


11 

12 

13 

14 

15 

16 

17 

Rating  Scores 

U1 

.23 

.20 

726 

.13 

.16 

.29 

U2 

.18 

.06 

.10 

.18 

.13 

.13 

.16 

U3 

•  31 

.22 

.26 

.34 

.29 

.22 

.26 

U4 

.23 

.14 

.15 

.26 

.16 

.2  3 

.23 

U5 

.15 

.16 

.15 

.25 

.23 

.09 

.17 

U6 

.26 

.11 

.17 

.29 

.19 

.17 

.25 

U7 

.12 

.10 

.21 

.30 

.17 

.06 

.24 

U8 

.25 

.06 

.11 

.25 

.18 

.20 

.24 

U9 

.30 

.18 

.18 

•  32 

.17 

.19 

.25 

U10 

.16 

.12 

.13 

.25 

.19 

.09 

.20 

U11 

.33 

.16 

.21 

.30 

.31 

.21 

.32 

U12 

.13 

.19 

.18 

.23 

.23 

.07 

.25 

U1 3 

.24 

.17 

.14 

.21 

.19 

.16 

.32 

1)14 

.31 

.19 

.19 

.32 

.22 

.21 

.29 

U15 

.24 

.13 

.10 

.24 

.23 

.19 

.35 

U16 

.34 

.17 

.31 

.31 

.23 

.30 

.28 

U17 

.16 

,11 

.09 

.27 

.19 

.21 

.16 

Mean 

Correlation  «  . 

20 

Distance  in 

in  ,26  .25 

U2 

.21 

.08 

.11 

.12 

.17 

.12 

.18 

U3 

.31 

.24 

.25 

.29 

.29 

.20 

.31 

U4 

.21 

.17 

.13 

.24 

.20 

.23 

.24 

U5 

.10 

.19 

.11 

.17 

.21 

.13 

.15 

U6 

.25 

.16 

.22 

.27 

.21 

.19 

.28 

U7 

.22 

.21 

.20 

.29 

.22 

.13 

.29 

U8 

.27 

.16 

.17 

.25 

.26 

.20 

.29 

U9 

.34 

.18 

.20 

.34 

.24 

.22 

.28 

U10 

.19 

.16 

.11 

.19 

.18 

.15 

.27 

U11 

.28 

.19 

.20 

.23 

.32 

.21 

.27 

U12 

.22 

.26 

.21 

.20 

.25 

.14 

.21 

U13 

.23 

.15 

.16 

.20 

.23 

.15 

.32 

1)14 

.37 

.29 

.24 

.32 

.30 

.29 

.29 

U15 

.18 

.17 

.10 

,21 

.20 

.21 

.26 

U16 

•  30 

.2? 

.31 

.26 

.26 

.26 

.30 

U 1 7 

.14 

.23 

.09 

.30 

.27 

.20 

.16 

Meen 

Correlation  ■  , 

.22 

Split-Plane  Manuevers 


Inter-Individual  Correlation  Matrix  for  Instructor  Pilots 
and  Instructor  Weapons  Systems  Officers 

II  through  17  are  individual  instructor  pilots 
W1  through  W4  are  individual  IWSOs 


Rating  Scores 


11 

12 

13 

IH 

15 

16 

17 

W1 

.37 

.37 

.Hi 

.40 

.45 

.38 

.38 

W2 

.21 

.HO 

.30 

.22 

.36 

.20 

.26 

W3 

.39 

.35 

.  H2 

.49 

.43 

.37 

.48 

wu 

.46 

.HO 

.54 

.46 

.42 

.48 

.47 

Mean  Correlation  a  .39 
Distance  in  Multidimensional  Space 


11 

12 

13 

14 

15 

16 

17 

W1 

.41 

.34 

.43 

.42 

.43 

.43 

.42 

W2 

.09 

.31 

.22 

.15 

.26 

.15 

.21 

W3 

.44 

.42 

.54 

.53 

.50 

.44 

.55 

W4 

.54 

.45 

.55 

.50 

.47 

.51 

.53 

Mean  Correlation  a  .HO 


-99- 


Split-Plane  Manuevers 

Inter-Individual  Correlation  Matrix  for  GPa  and  UPTa 

G1  through  G9  are  Individual  Guard  pilota 
U1  through  U 1 7  are  individual  Undergraduate  pilota 


G1 

G2 

C3 

04 

05 

06 

G7 

C8 

09 

Rating  Soorea 

U1 

.19 

.17 

.19 

.21 

.28 

.22 

.17 

.21 

.21 

U2 

.24 

.23 

.18 

.29 

.15 

.26 

.18 

.28 

.12 

U3 

.23 

.33 

.29 

.35 

.29 

.28 

.30 

.30 

.21 

U4 

.17 

.24 

.19 

.26 

.26 

.25 

.13 

.26 

.21 

U5 

.09 

.21 

.19 

.21 

.19 

.22 

.13 

.20 

.15 

U6 

.23 

.21 

.29 

.30 

.21 

.23 

.18 

.30 

.23 

U7 

.15 

.27 

.22 

.27 

.29 

.22 

.23 

.09 

.17 

U8 

.18 

.24 

.26 

.28 

.19 

.15 

.19 

.28 

.21 

U9 

.23 

.26 

.29 

.28 

.28 

.31 

.22 

.28 

.21 

U10 

.19 

.26 

.19 

.25 

.26 

.28 

.16 

.20 

.19 

U11 

.27 

.31 

.27 

.38 

.36 

.33 

.28 

.35 

.34 

U12 

.15 

.24 

.25 

.26 

.29 

.23 

.16 

.14 

.18 

U 1 3 

.31 

.28 

.30 

.29 

.25 

.27 

.22 

.22 

.21 

U14 

.17 

.23 

.27 

.35 

.30 

.32 

.23 

.25 

.20 

U15 

.29 

.23 

.25 

.29 

.28 

.26 

.20 

.23 

.24 

U 1 6 

.20 

.40 

.29 

.29 

.26 

.28 

.28 

.29 

.26 

017 

.14 

.20 

.22 

.28 

.16 

.22 

.16 

.18 

.18 

Mean  Correlation  »  .24 


Dlatancea  in  Multldiinenaional  Space 


U1 

Tai r 

.19 

.22” 

.21* 

.27 

.23 

.32 

.27 

.25 

U2 

.21 

.23 

.13 

.24 

.16 

.20 

.23 

.23 

.10 

U3 

.21 

.32 

.31 

.32 

.28 

.32 

.32 

.30 

.22 

U4 

.22 

.28 

.18 

.24 

.23 

.25 

.14 

.29 

.17 

U5 

.13 

.12 

.19 

.17 

.15 

.21 

.17 

.16 

.09 

U6 

.25 

.20 

.33 

.37 

.20 

.24 

.23 

.27 

.22 

U7 

.15 

.30 

.28 

.33 

.32 

.22 

.31 

.16 

.25 

U8 

.21 

.27 

.23 

.32 

.17 

.15 

.23 

.33 

.19 

U9 

•  30 

.24 

.28 

.27 

.23 

.27 

.30 

.24 

.22 

U10 

.25 

.28 

.25 

.33 

.26 

.25 

.28 

.28 

.24 

U11 

.28 

.28 

.25 

.33 

.39 

•  30 

.32 

.36 

.24 

U12 

.18 

.21 

.22 

.26 

.16 

.17 

.22 

.11 

.19 

U1 3 

.23 

.27 

.23 

.28 

.23 

.29 

.26 

.25 

.12 

U14 

.22 

.25 

.27 

.38 

.28 

.28 

.28 

.28 

.22 

U15 

.26 

.20 

.23 

.29 

.18 

.21 

.19 

.30 

.19 

U 1 6 

.24 

.46 

.31 

.33 

.30 

.26 

.26 

.30 

.24 

U 1 7 

.14 

.19 

.18 

.31 

.10 

.17 

.23 

.15 

.18 

Mean  Correlation  c  .24 
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Split-Plane  Manuevers 

Inter-Individual  Correlation  Matrix  for  Guard  Pilots 
and  Inatruotor  Weapons  Systems  Officers 

G1  through  G9  are  individual  Guard  pilots 
W1  through  W4  are  individual  IWSOs 


Rating  Soores 


G1 

G2 

G3 

04 

G5 

G6 

07 

08 

09 

W1 

.39 

•  34 

.46 

.34 

.31 

.40 

.33 

.25 

.46 

W2 

.30 

.23 

.25 

.15 

.18 

.21 

.17 

.10 

.25 

W3 

.38 

.40 

.44 

.24 

.33 

.23 

.41 

.22 

.38 

W4 

.*<5 

.42 

.44 

.23 

.30 

.26 

.39 

.24 

.45 

Mean  Correlation  « 

.31 

Distances 

in  Multidimensional 

Space 

G1 

02 

03 

04 

05 

06 

07 

08 

G9 

W1 

.36 

.35 

.48 

.44 

.37 

.36 

.38 

.22 

.49 

W2 

.18 

.21 

.19 

.08 

.12 

.09 

.13 

.04 

.24 

W3 

.39 

.45 

.55 

.35 

.39 

.31 

.49 

.27 

.48 

WM 

.49 

.48 

.48 

.31 

.39 

.32 

.44 

.28 

.49 

Mean  Correlation  *  .34 


101- 


Split-Plane  Hanuevers 

Inter-Individual  Correlation  Matrix  for  UPTa  and  IWSOs 

U1  through  U 1 7  are  individual  undergraduate  pilots 
W1  through  W4  are  Individual  IWSOs 


W1  W2  W3  W4 


Rating  3 gores 


01 

.i  r 

.07 

.26 

".19 

U2 

.09 

.01 

.14 

.08 

U3 

.26 

.10 

.22 

.25 

04 

.20 

.03 

.23 

.22 

05 

.23 

.11 

.15 

.11 

06 

.22 

.10 

.23 

.22 

07 

.2? 

.10 

.26 

.21 

08 

.16 

.04 

.26 

.24 

09 

.18 

.04 

.24 

.25 

010 

.21 

.07 

.20 

.16 

Oil 

.35 

.12 

.30 

.29 

012 

.22 

.19 

.21 

.13 

013 

.27 

.11 

.27 

.24 

U14 

.30 

.08 

.33 

.19 

015 

.17 

.09 

.19 

.17 

016 

.30 

.17 

.29 

.25 

017 

,?9. 

.04 

.15 

.15 

Mean  Correlation  ■  .18 


Dlatanoea  in  Multldimenaional  Spaoe 


01 

.23 

.12 

.29 

.26 

02 

.14 

.03 

.16 

.10 

03 

.27 

.07 

.32 

.26 

04 

.19 

.10 

.26 

.22 

05 

.25 

.12 

.19 

.10 

U6 

.26 

.05 

.37 

.23 

07 

.33 

.12 

.36 

.28 

08 

.14 

.01 

.29 

.25 

U9 

.22 

.05 

.32 

.32 

010 

.25 

.05 

.29 

.20 

011 

.29 

.11 

.27 

.32 

012 

.19 

.17 

.20 

,24 

013 

.21 

.09 

.31 

.29 

014 

.27 

.06 

.35 

.23 

015 

.12 

.04 

.17 

.17 

016 

.29 

.17 

.30 

.27 

017 

.22 

.08 

.20 

.11 

Mean  Correlation  a  .20 
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Low-Angle  Strafe 

Inter-Individual  Correlation  Matrix 
II  through  16  are  individual  instructor  pilots 


Rating  Scores 


11 

12 

13 

14 

15 

16 

11 

m 

.49 

.39 

.57 

.49 

.41 

12 

.49 

- 

.40 

.50 

.42 

.39 

13 

•  39 

.40 

- 

.47 

.36 

.31 

14 

.57 

.50 

.47 

- 

.53 

.45 

13 

.49 

.42 

.36 

.53 

. 

.42 

16 

.41 

.39 

.31 

.45 

.42 

- 

Mean  Correlation  a  .44 


Dlatanoea  in  Multidimensional  Space 


11 

12 

13 

14 

15 

16 

11 

.57 

.45 

.64 

.51 

.44 

12 

.57 

. 

.44 

.57 

.41 

.50 

13 

.45 

.44 

. 

.54 

.44 

.38 

14 

.64 

.57 

.54 

m 

.53 

.49 

15 

.51 

.41 

.44 

.53 

m 

.44 

16 

.44 

.50 

.38 

.49 

.44 

- 

Mean  Correlation  ■  .49 


*  •  w'jrr'5’" 


Low-Angle  Strafft 

Inter-Individual  Correlation  Matrix 
U1  through  U16  are  individual  UPTs  • 


01 

02 

03 

04 

05 

06 

07 

08 

U9 

U10 

011 

012 

cr 

U> 

014 

015 

016 

Rating 

Scores 

ui 

.23 

.27 

.18 

.22 

.30 

.25 

.28 

.24 

.35 

.29 

.  29 

.?9 

.25 

.18 

.25 

02 

.23 

.40 

•  31 

.33 

.32 

.33 

.39 

.34 

.41 

.30 

.20 

.41 

.29 

.29 

.40 

U3 

.27 

.no 

.42 

.24 

.29 

.32 

.47 

.37 

.51 

.38 

.21 

.38 

.27 

.37 

.37 

on 

.18 

.31 

.42 

- 

.25 

.27 

.27 

•  34 

.25 

.30 

.28 

.22 

.31 

.25 

.37 

.32 

05 

.22 

.33 

.24 

.25 

- 

.23 

.37 

•  31 

.23 

.32 

.09 

.24 

.21 

.26 

.27 

.32 

06 

•  30 

.32 

.29 

.27 

•  23 

- 

.40 

.45 

.44 

.47 

.25 

.36 

.31 

.31 

.33 

.32 

07 

.25 

.33 

.32 

.27 

•  37 

.40 

- 

.31 

.41 

.47 

.16 

.37 

.39 

.29 

.32 

.37 

08 

.28 

.39 

.47 

.34 

•  31 

.45 

.31 

- 

.35 

.49 

.31 

.32 

.37 

.33 

.40 

.41 

U9 

.2n 

•  3M 

.37 

.25 

.23 

.44 

.41 

.35 

- 

.42 

.24 

.35 

.36 

•  32 

.36 

.36 

U10 

.35 

.m 

.51 

.30 

.32 

.47 

.47 

.49 

.42 

- 

•?9 

.43 

.43 

.37 

.40 

.46 

011 

.29 

•  30 

.38 

.28 

.09 

.25 

.16 

.31 

.24 

.29 

.11 

.27 

.25 

.18 

.21 

012 

.29 

.20 

.21 

.22 

.24 

.36 

.37 

.32 

.35 

.43 

.11 

- 

.19 

.21 

.24 

.32 

U 1 3 

.29 

.41 

.38 

.31 

.21 

.31 

.39 

•  37 

.36 

.43 

.27 

.19 

- 

.29 

.34 

.39 

u  i  n 

.25 

.29 

.27 

.25 

.26 

.31 

.29 

.33 

.32 

.37 

.25 

.21 

.29 

- 

.29 

.25 

U15 

.18 

.29 

.37 

.37 

.27 

.33 

.32 

.40 

.36 

.40 

.18 

.24 

.34 

.29 

m 

.38 

016 

.25 

.40 

.37 

.32 

.32 

.32 

.37 

.41 

.36 

.46 

.21 

.32 

.39 

.25 

.38 

- 

Mean  Correlation  a 

.32 

Diatanoes 

in  Multidimensional 

Spaoe 

U1 

.20 

.17 

.14 

.29 

,27 

.22 

.34 

.18 

.31 

.17 

.22 

.26 

.24 

.25 

.24 

02 

.20 

m 

.41 

.29 

.31 

.40 

.36 

.47 

.39 

.41 

.20 

.17 

.41 

.38 

.27 

.49 

03 

.17 

.41 

.40 

.27 

.33 

.28 

.45 

.23 

.45 

.25 

.20 

.36 

.27 

.32 

.38 

on 

.in 

.29 

.40 

- 

.15 

.26 

.17 

.28 

.15 

.22 

.14 

.20 

.19 

.19 

.27 

.20 

05 

.29 

.31 

.27 

.15 

- 

.28 

.36 

.35 

.23 

.35 

.11 

.19 

.29 

.29 

.28 

.37 

U6 

,27 

.40 

.33 

.26 

.28 

.44 

.50 

.56 

.51 

.22 

.41 

.33 

.40 

.31 

.36 

07 

.22 

.36 

.28 

.17 

.36 

.44 

.39 

.43 

.53 

.13 

•  33 

.45 

.32 

.31 

.46 

08 

.3n 

.47 

.45 

.28 

.35 

.50 

.39 

- 

.33 

.56 

.28 

.30 

.39 

.41 

.44 

.59 

09 

.18 

.39 

.23 

.15 

.23 

.56 

.43 

.33 

.46 

.26 

.34 

.40 

.40 

.31 

.36 

010 

.31 

.41 

.45 

.22 

.35 

.51 

.53 

.56 

.46 

- 

.24 

.37 

.47 

.48 

.42 

.54 

U11 

.17 

.20 

.25 

.14 

.11 

.22 

.13 

.28 

.26 

.24 

- 

.08 

.13 

.22 

.14 

.32 

012 

.22 

.17 

.20 

.20 

.19 

.41 

.33 

.30 

.34 

.37 

.08 

- 

.21 

.28 

.20 

.31 

013 

.26 

.41 

.36 

.19 

.29 

.33 

.45 

.39 

.40 

.47 

.13 

.21 

- 

.32 

.31 

.44 

om 

.2n 

.38 

.27 

.19 

.29 

.40 

.32 

.41 

.40 

.48 

.22 

.28 

.32 

- 

.29 

.38 

015 

.25 

.27 

.32 

.27 

.28 

.31 

.31 

.44 

.31 

.42 

.14 

.20 

.31 

.29 

- 

.35 

016 

.2n 

.49 

.38 

.20 

.37 

.36 

.46 

.59 

.36 

.54 

.32 

.31 

.44 

.38 

.35 

- 

Mean  Correlation  a  .32 
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F 


Low-Angle  Strafe 

Inter-Individual  Correlation  Matrix 
W1  through  W7  are  individual  weapons  systems  offioers 


Rating  Spores 


W1 

W2 

M3 

W4 

W5 

W6 

W7 

W1 

m 

.43 

.49 

.34 

.09 

.56 

.46 

W2 

.*•3 

- 

.49 

.37 

.19 

.50 

.44 

W3 

.49 

.49 

- 

.39 

.15 

.52 

.45 

wu 

.34 

.37 

.39 

- 

.18 

.39 

.29 

W5 

.09 

.19 

.15 

.18 

- 

.17 

.12 

W6 

.56 

.50 

.52 

.39 

.17 

- 

.50 

W7 

.46 

.44 

.45 

.29 

.12 

.50 

m 

Mean 

Correlation 

i  «  . 

36 

Dlatanoea 

in  Multidimensional 

Space 

Ml 

W2 

W3 

W4 

M5 

W6 

W7 

W1 

.56 

.43 

.17 

.46 

.51 

.52 

W2 

.56 

- 

.51 

.12 

.55 

.52 

.50 

W3 

.43 

.51 

- 

.11 

.45 

.42 

.39 

W4 

.17 

.12 

.11 

m 

.16 

.11 

.17 

W5 

.46 

.55 

.45 

,16 

m 

.53 

.49 

W6 

.51 

.52 

.42 

.11 

•53 

- 

.54 

W7 

.52 

.50 

.39 

.17 

.49 

.54 

- 

Mean  Correlation  ■  ,39 


105- 


Low-Angle  Strafe 

Inter-Individual  Correlation  Matrix 

II  through  16  are  Individual  instructor  pilots 
U1  through  016  are  individual  undergraduate  pilots 

II  12  13  14  15  16 


Rating  Soorea 


U1 

.19 

.16 

.18 

.15 

.09 

.21 

02 

.17 

.21 

.28 

.25 

.22 

.23 

03 

.11 

.09 

.20 

.16 

.13 

.11 

04 

.12 

.09 

.IB 

.14 

.09 

.14 

U5 

.13 

.16 

.23 

.17 

.11 

.16 

U6 

.21 

.22 

.22 

.21 

.28 

.32 

U7 

.14 

.17 

.31 

.21 

.26 

.22 

1)8 

.23 

.19 

.26 

.22 

.23 

.27 

U9 

.17 

.14 

.33 

.18 

.29 

.27 

010 

.30 

.29 

.41 

.32 

.32 

.27 

on 

,06 

.03 

.11 

.04 

.00 

.11 

U12 

.29 

.20 

.26 

.28 

.26 

.28 

013 

.21 

.21 

.25 

.21 

.32 

.22 

014 

.13 

.14 

■  32 

.20 

.23 

.21 

015 

.10 

.18 

.35 

.14 

.18 

.23 

U16 

.15 

.13 

.30 

.24 

.23 

.13 

Mean  Correlation  ■  .20 


Pletanoea  jjj  Multidimensional  Spaoe 


ui 

.22 

.24 

.20 

.20 

.17 

.19 

U2 

.23 

.25 

.30 

.23 

.29 

.21 

U3 

.10 

.12 

.23 

.18 

.11 

.06 

U4 

.11 

.11 

.19 

.13 

.09 

.05 

05 

.12 

.20 

.27 

.21 

.15 

.18 

U6 

.21 

.19 

.25 

.24 

.30 

.30 

U7 

.19 

.19 

.35 

.25 

.29 

.21 

U8 

.22 

.22 

.34 

.28 

.30 

.27 

U9 

.19 

.17 

.28 

.18 

.26 

.28 

010 

.29 

.31 

.45 

.35 

.34 

.26 

U1 1 

.08 

.04 

.15 

.08 

.03 

.04 

012 

.37 

.29 

.31 

.38 

•  34 

.30 

U13 

.24 

.18 

.24 

.21 

.27 

.15 

U14 

.21 

.20 

.37 

.25 

•  34 

.25 

U 1 5 

.19 

.16 

.36 

.21 

.20 

.25 

016 

.21 

.19 

.34 

.30 

.23 

.11 

Mean  Correlation  ■  .22 


106- 


Low-Angle  Strafe 

Inter-Individual  Correlation  Matrix 


II  through  16  are  individual  inatruotor  pilots 
W1  through  W7  are  individual  weapons  systems  officers 


Rating  Scores 


11 

12 

13 

14 

ID 

16 

W1 

.40 

.no 

.42 

.52 

.54 

.44 

W2 

.36 

.32 

.nn 

.44 

.45 

.34 

W3 

.50 

.n3 

.50 

.54 

.45 

.48 

wn 

.36 

.36 

.42 

.40 

•  39 

.38 

W3 

.05 

.ii 

.18 

.05 

.05 

.16 

W6 

.ns 

.45 

.51 

.61 

.46 

.45 

W7 

.ns 

.45 

.34 

.51 

.46 

.47 

Mean  Correlation  ■  .39 


Low- Angle  Strafe 

Inter-Individual  Correlation  Matrix 

U1  through  1116  are  individual  undergraduate  pilota 
W1  through  W7  are  individual  weapons  systems  offioers 

W1  W2  W3  W«4  W5  W6  W7 


Retina  Soorea 


U1 

.20 

.24 

.21 

.21 

.13 

.22 

.16 

1)2 

.26 

.36 

.24 

.30 

.18 

.31 

.25 

U3 

.15 

.27 

.16 

.23 

.20 

.26 

.11 

U4 

.14 

.22 

.15 

.16 

.23 

.23 

.09 

U5 

.21 

.25 

.20 

.21 

.16 

.19 

.16 

U6 

.25 

.36 

.30 

.23 

.18 

.36 

.29 

U7 

.31 

.40 

.21 

.30 

.15 

.33 

.22 

U8 

.28 

.37 

.25 

.25 

.19 

.35 

.28 

U9 

.26 

.28 

.22 

.31 

.14 

.33 

.23 

010 

.35 

.45 

.41 

.33 

.20 

.44 

.28 

Oil 

.06 

.08 

.13 

.10 

.07 

.15 

.03 

(112 

.28 

.33 

.33 

.35 

.10 

.30 

.24 

013 

.33 

.33 

.20 

.25 

.09 

.32 

.21 

(114 

.27 

.30 

.22 

.23 

.17 

.30 

.22 

U 1 5 

.26 

.26 

.17 

.21 

.26 

.30 

.14 

016 

.26 

.32 

.26 

.31 

.14 

.29 

.18 

Mean  Correlation  •  .24 


Diatanoea  .in  Multidimensional  Spaoo 


01 

.23 

.15 

.21 

.15 

.25 

.26 

.30 

(12 

.41 

.23 

.29 

.16 

.26 

.28 

.35 

03 

.25 

.18 

.20 

.08 

.14 

.21 

.19 

04 

.20 

.09 

.12 

.09 

.07 

.18 

.11 

05 

.30 

.25 

.22 

.12 

.29 

.29 

.26 

06 

.35 

.26 

.28 

.11 

.25 

.37 

.28 

07 

.33 

.25 

.26 

.06 

.31 

.39 

.25 

08 

.46 

.28 

.31 

.12 

.37 

.36 

.32 

09 

.27 

.23 

.32 

.11 

.25 

.36 

.25 

010 

.45 

.41 

.34 

.10 

.37 

.50 

.34 

011 

.05 

.17 

.21 

.01 

.08 

.10 

.08 

012 

.31 

.38 

.39 

.06 

.33 

.42 

.26 

013 

.30 

.22 

.21 

.03 

.27 

.28 

.24 

014 

.32 

.26 

.28 

.15 

.29 

.32 

.31 

015 

.33 

.20 

.28 

.23 

.29 

.35 

.22 

016 

.40 

.31 

.38 

.05 

.34 

.34 

.26 

Mean  Correlation  ■  .25 


108' 


